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ABSTRACT 



The use and acceptance by Navy personnel of 16 major 
training devices were studied in relation to: (1) situa- 
tional factors affecting training; (2) simulation charac- 
teristics of the trainer; (3) instructional characteristics 
of the trainers; reliability of the trainers; formal and 
informal communications regarding trainer capabilities; and 
level of experience of the users in the systems simulated 
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included students, instructors, administrative, and main- 
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An acceptance profiling technique was developed that 
appears to be highly diagnostic of the reasons for accep- 
tance or rejection of particular trainers. It was evident 
that both highly accepted and seriously rejected trainers 
were represented in the sample. Methods for increasing 
trainer acceptance are outlined in terms of improvement in 
specific areas of simulation; improved software; greater 
qualifications for instructors; improved evaluation of per- 
formance; and improved understanding of the purpose, capa- 
bilities, and limitations of trainers by the users. The 
merits of continuing studies of trainer acceptance and the 
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FOREWORD 



An important determinant of the effectiveness of training device 
systems—one that has received relatively little research attention — 
is the attitude of the trainees, instructors, and managers toward 
them. 

This study was undertaken to identify the factors leading to the 
acceptance or rejection of training devices by their users and to 
determine the extent to which these factors were present in 
"effective" and "ineffective" training devices. Future training 
devices, including the associated training materials and recommenda- 
tions for patterns of use, can be developed with a knowledge of 
these facts on hand, A second objective was to develop a 
methodology which could be used in the future by the Naval Training 
Device Center to assess the acceptability of training devices. 

Factors found to influence acceptance are subsumed under six main 
categories. These are: (l) simulation factors, (2) specific 
trainer features, (3) characteristics of users, (4) characteristics 
of instruction, (5) patterns of use, and (6) the manner in which 
devices are introduced to the user. 

Although no trainer is totally "acceptable" or "\macceptable," 
improvement in the above listed categories will result in higher 
acceptance of trainers with the consequence of 'improved training. 

Specific recommendations to increase acceptance and improve 
methods of utilizing complex trainers include: (l) greater in- 
vestments in training software, (2) providing better technical 
documentation, (3) providing for higher levels of fidelity in 
the "feel" of the simulated vehicle, (4) developing uniform and 
high standards for instructors, and (5) developing a "trainer 
advocate" for devices representing a major investment. 

The acceptance profiling technique developed for this study 
should be applied periodically to all complex trainers so that 
a body of knowledge is developed that will be useful for design 
decisions. 




LEONARD E, RYAN ^ 
Research Psychologist 
Naval Training Device Center 
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SECTION I 
INTRODUCTION 

The critical role of training devices in developing and 
maintaining the skills of Navy personnel is accepted by vir- 
tually everyone. Recognition of the need for a training 
device, however, does not automatically insure a high level 
of acceptance of the device by its users • Many factors op- 
erate, some perhaps obvious and some quite subtle, to deter- 
mine the level of acceptance and method of use of training 
devices by military personnel. In extreme cases, strong pat- 
terns of rejection can develop that can defeat the very best 
of intentions and large-scale investments in training systems. 

The problem is not peculiar to training devices. The 
Navy has become increasingly concerned about the potential 
degradation of systems performance associated with non- 
acceptance of various kinds of Oj ^rational equipment by the 
personnel who must maintain and use it. Several studies 
have shown that in the hands of the users, new equipment 
often fails to achieve its full potential performance capa- 
bility. It is obviously in the interest of all to identify 
the factors that operate in the acceptance or rejection of 
Navy hardware systems and to try to develop techniques whereby 
acceptance can be maximized. 

In the economic world, the process of acceptance has been 
a major concern of every manufacturer of consumer goods--how 
does one gain acceptance on the part of the public to which 
he wishes to sell his product? In the marketplace, the conse- 
quences of consumer resistance are immediate and economically 
severe. In the military, where there usually is no competi- 
tion (i.e., the user has no choice), the consequences are more 
subtle but very likely just as serious. 

It was the objective of the study reported here to (1) 
identify the factors leading to the acceptance (or rejection) 
of training devices by their users; and (2) to determine the 
extent to which these acceptance factors were present in 
"effective'' and "ineffective" training devices. 

SOME BACKGROUND CONCERNING THE PROCESS OF ACCEPTANCE 

The processes of acceptance and rejection have long been 
studied from the academic viewpoint by the social scientists 
concerned with attitude formation and change. The appropri- 
ate literature was carefully reviewed in an effort to develop 
hypotheses that might be pertinent to the acceptance of train- 
ing devices. Within the literature there was also a small 



11 



NAVTRADEVCEN 70-C-0276-1 



number of studies directly concerned with the acceptance of 
hardware developments by Navy personnel. In one of the 
earliest such studies, Simpson and Parker (18) performed a 
field study of the acceptance of the ASPECT sonar system by 
fleet personnel. The investigation was concerned both with 
the operational use of this equipment and the degree to which 
it had been maintained in good operational order. It was 
found that the equipment on board over half of the ships 
visited was either incapable of producing target signals at 
all, produced signals of very poor quality, or was so severely 
miscalibrated that proper operation was impossible. Simpson 
and Parker suggested that the principal problem lay in the 
fact that a lack of understanding jf the purpose and operation 
of the device had engendered biases against it. These biases 
led to insufficient maintenance, thus reducing potential 
operational effectiveness, which in turn further strengthened 
the biases, and so on, in a never-ending cycle. 

A number of investigators have stressed the importance 
of firsthand personal experience, in contrast to the simple 
presentation of factual information, in promoting the ac- 
ceptance of new equipment. Berger et al . (1) suggested that 
the importance of personal trial was the result of the oppor- 
tunity to overcome negative attitudes based on unf ami liari ty , 
to compare one's own performance using the new equipment with 
that using older equipment, to build confidence in how to 
use the equipment, and finally to provide an opportunity to 
communicate attitudes and ideas about the equipment to one ' s 
peers . 

An important incidental finding of Berger's study was 
that attitudes toward new equipment may change over time 
without any additional firsthand experience, as a result of 
informal communication. A period of initial resistance to 
innovations may well be overcome, in time, provided that 
the proper provisions for trial have been made. If not, 
negative attitudes become stabilized and possibly increase 
in strength with time. 

There is little objective documentation of user accep- 
tance problems with respect to training devices. However, 
in an earlier NTDC study (20), a committee of engineers, 
training specialists, and maintenance personnel investigated 
the use of two operational flight trainers. The trainers 
were studied from the viewpoint of how effectively they were 
utilized by both training commands and the fleet. These in- 
vestigators identified significant differences between design 
capability and the manner in which the trainers were actually 
used. They found that neither of the operational flight 
trainers was in a condition that permitted use of all of its 
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capabilities, either in a single problem , or in a series of 
problems. It was found that despite their other capabilities, 
the OFT's were used almost exclusively for cockpit familiar- 
ization training in normal and emergency procedures. They 
reported that squadron training officers for the most part 
had little confidence in the trainers, and did not as a rule 
make OFT instruction a part of the training syllabus. It 
appeared that some pilots had a **generali zed dislike** for 
simulators as a result of early negative experiences. It was 
concluded that the problem of OFT utilization might be prin- 
cipally one of educating the users . 

In many respects, the problem of optimizing the utiliza- 
tion of a training device is similar to that of maximizing 
the demand for a consumer product. The ultimate user of a 
trainer mus*: be convinced of its worth, and educated con- 
cerning its capabilities and limitations . Apathetic accep- 
tance may lead to poor methods of utilization which, in 
turn, may eventually result in actual rejection of the train- 
ing device. The latter can also result from overselling or 
misinformation concerning a trainer*s capabilities. 

In a recent study of the acceptance of Navy equipment, 
Mecherikoff and Mackie (11) showed that the effects of de- 
sign deficiencies can become accentuated through personality 
dynamics. One of the most important dynamic reactions that 
clearly led to rejection was the **not invented here** phenom- 
enon. This refers to the formation of negative attitudes 
that can result from the absence of personal involvement in 
a technical innovation in one*s own area of expertise. Any 
deficiency in the design of the innovation may be magnified 
in importance by the **not invented here** reaction. 

Another factor identified in this study as important to 
the acceptance process has been clearly emphasized in the 
extensive literature on attitude formation and change. Atti- 
tudes toward new equipment or other innovations are critical ly 
determined by the ** change advocate** --some individual who 
functions, formally or informally, in the role of initial com- 
municator concerning the advantages and capabilities of the 
device to the potential users. The qualifications and other 
characteristics of the change advocate, real or perceived, 
are critically important to the development of acceptance, 
particularly if the innovation is in an area where the poten- 
tial users consider themselves experts. Such factors as the 
change advocate *s credibility, prestige, perceived motivation, 
relationship to the users, apparent impartiality, intent to 
influence, methods of handling criticism, etc., are factors 
that significantly affect his success in promoting the accep- 
tance of new equipment. 
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While the importance of the change advocate has been 
known to social scientists for many years, it is apparent 
that there has been no systematic attempt to employ well 
qualified advocates for the introduction of new equipment 
into the Navy, whether that equipment be operational or 
training equipment. In fact, in many cases, the role of 
change advocate is never filled at all, with the not unpre- 
dictable consequence of disuse or misuse of the equipment. 
In some cases, the role of change advocate falls by default 
on petty officers whose primary responsibility toward the 
training device is its operation and maintenance. In a few 
instances, the role of change advocate has been unofficially 
assumed by some highly motivated individual who recognizes 
the need to solve a training problem and is willing to do 
something about it, however sophisticated his approach may 
be. But in many cases, there is no qualified advocate at 
all, official or otherwise. 

When a training device is designed, the designers nake 
judgments and decisions based on their understanding of 
important parameters of the total situation. These hopefully 
include : 

1. The deficiencies of present devices (procedures 
or systems ) . 

2. How learning occurs. 

3. How the device will fit into an existing 
curriculum and what changes in curriculum 
may be required. 

4. The range of usefulness of the device (specifi- 
cally what it is intended to accomplish and 
what it is not intended to accomplish) . 

It seems likely that the designers themselves may not 
be fully aware of the extent to which they are making such 
decisions; certainly, the assumption is doubtful that the 
eventual users understand all of these factors in the same 
way that the designer does, or that the hardware itself will 
somehow convey its purposes. 



McClelland (10) in his extensive analysis of the pro- 
cesses of effecting change, has identified a number of 
fallacious propositions that clearly appear to be pertinent 
to the introduction of training 



devices to 
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A good product will succeed on its own merits 
(Don't you believe it.) 
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2. The introduction of a new device is a final 
act. No further attention is required . (In 
the absence of a f ormul a for maintenance and 
feedback, many training innovations are gath- 
ering dust and teachers and managers have 
reverted to former practices.) 

3. There is an orderly process from research to 
development to use. (The fallacy of this 
proposition has been emphasized by numerous 
investigators . ) 

Quesada (15) has identified five criteria that appear to 
play significant roles in the acceptance of innovations. 
These have been paraphrased with reference to training devices. 

1. Relative Advantage . The degree to which the 
training device is perceived as being superior 
to the one it succeeds. 



2. Compatibility. The degree to which the device 
is perceived as consistent with operational 
requirements , equipment , and past experience . 

3 . Complexity . The degree to which the device is 
relatively difficult or easy to understand and 
use . 

4. Visibility . The degree to which the results 
of using the devi ce may be transmitted to 
others in a way that is easily seen or demon- 
strated . 



5. Divisibility . The degree to which the device 
may be tried on a limited basis to gain first- 
hand personal experience. 

It seems likely that all of these criteria apply, in one 
way or another, in determining the user's acceptance of Navy 
equipment and training devices. 

Quesada also distinguishes between three types of de- 
cisions in the acceptance process. An ''authority decision" 
results when those high in the power structure require adop- 
tion of an innovation. A "contingent decision" permits the 
individual to adopt or reject the innovation but only after 
an enabling decision is made by the organization. Finally, 
a "collective decision" occurs when individuals in a particu- 
lar group participate in the verdict concerning a given 
innovation. Decisions of all these types may be made in 
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connection with an indi/iduai*s use of a training device. 
They need not be mutualiy exclusive. An authority decision 
may be made that requires periodic use of the device, but 
this in no way prevents independent individual or group de- 
cisions concerning the value of using the device. Clearly, 
true accept an ce mus t be a function of the ''collective dc- 
cision** for it is the decision at the user level that even- 
tually determines whether the device is used effectively or 
not . 

Chin (3) has a similar trichotomy of what he calls 
strategies for change. The ''power approach" involves com- 
pliance and submission with a limiting of alternatives and 
shaping of consequences by those in authoritative positions. 
Chin points out that this may be counterproductive of the 
desired acceptance pattern. A second strategy is called 
"normative-reeducat ive . " This approach depends on the role 
of values in a group with which the innovation is identified. 
An example of this approach is "sensitivity training." The 
third is the "empirical-rational" approach which depends 
heavily on demonstration of the relative merits of the in- 
novat i on . 

In his studies of factors influencing the acceptance of 
planned changes in other cultures, Niehoff (13) has studied 
the transfer of innovations to new cultural groups. A number 
of conditions were identified that helped expedite this 
process : (1) the in no vat ion should be compatible with cul- 
tural patterns of the recipient group (the amount of new 
behavior required, and old behavior given up should be mini- 
mal) ; (2) the innovation should meet existing or felt needs 
of the recipient, probably those they have tried to solve 
through their own efforts; (3) the innovation should provide 
practical (economic) benefits; (4) the strategy of introduc- 
tion should involve adapting to and working through the local 
cultural patterns, particularly the pattern of local leader- 
ship; (5) channels of communication should be established 
through a change agent who provides for an efficient two-way 
flow of information (feedback from the recipients to the 
change agent is particularly vital); (6) the recipients 
should be involved in the introduction process through full 
participation; (7) the change agent should be flexible in his 
strategies; (8) the change agent should establish patterns of 
maintenance (of the innovation) among the recipients so that 
the innovation can be continued when his influence is with- 
drawn . 

When it comes to innovations in training systems, it 
seems likely that the attitudes of the instructor (as well 
as his competence) can be critical ly import an t to user 
acceptance. The literature on teacher attitudes toward 
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various approaches to instruction and training media is 
voluminous. One recent study, which is perhaps typical, was 
reported by Finch, et at., (6) who studied the attitudes of 
100 instructors in vocat i onal-tec.\nical schools to determine 
interre lat ionships among teacher attitudes toward resources , 
resource utilization and avai labi lity . The results indicated 
that "the teacher group generally had a more favorable atti- 
tude toward ' traditional ' instructional resources as opposed 
to 'progressive* materials. The traditional materials were 
used more often and were more readily available. Relation- 
ships between attitude and use and between availability and 
use were general ly positive and significant." A factor 
analysis of the attitude variables revealed that "teachers 
may view instructional resources in accordance with their 
personal involvement with resource preparation, selection, 
presentation, and application. The results indicate that 
pre -service and in-service experience should be provided to 
acquaint teachers with new resources." 

On a related point, Toye (19) makes the apparently not 
so obvious observation that a training program has to be 
capable of working not only when it is installed but of con- 
tinuing to work despite organizational changes and accidents 
that beset any operation. "The literature on training and 
training methods rarely mentions how a training scheme works 
after its initial validation." As will be seen, this is a 
significant observation for the continued successful use of 
Navy training devices. 
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SECTION II 

SOME '^'^ECIFIC HYPOTHESES CONCERNING ACCEPTANCE 

In their study of attitudinal factors influencing the 
acceptance of new equipment in the Navy, Mecherikoff and 
Mackie (op. ait,) attempted to summari ze all of the in- 
fluencing factors that appear to operate in the pattern of 
acceptance or rejection. These factors relate in part to the 
hardware, in part to the processes of introducing and promot- 
ing the device (advocacy). Not all of the factors were 
considered equally important or even applicable in every case< 
However, the list appears fairly exhaustive of the factors 
that should be considered in the course of introducing any 
new equipment. In the reproduction of this list, the state- 
ments have been paraphrased as they might apply specifically 
to training devices. 

HARDWARE OR SYSTEM FACTORS 

Conceptual 

1. Agreement on definition of the need or require- 
ment for a particular training device. 

2. Various engineering approaches to device design* 
Physical Factors 

1. Equipment reliability and maintainability. 

2. Problems of mismatch with the operating en- 
vironment . 

3. Problems of mismatch with the capabilities of 
user personnel. 

4. Possibilities of mismatch with other elements 
of the training system. 

Psychological Factors 

1. Reaction to the appearance of the training device. 

2. Perception of its "fit" into the training en- 
vironment . 

3. Reactions to de lays in delivery (the device 
itself, components, spare parts, software). 
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4. Opinions formed on the basis of hearsay and 
rumor . 

5. Opinions formed on the basis of limited ex- 
perience with the device. 

Support 

1. Documentation of the purposes and functions 
of the device. 

2. Documentation on how the device should be 
operated . 

3. Documentation of technical specifications and 
maintenance data. 

ADVOCACY FACTORS 

Consideration of: 

1. What different kinds of persons or groups will 
use the device and what are their relevant 
characteristics . 

2. What communication channels were provided for 
user inputs during the design phases. 

3. What means were provided to detect and resolve 
differences in approach or philosophy concerning 
the design of the device. 

4. What users likely will want to know about the 
device . 

a. Overall purpose. 

b. Direct and indirect benefits to themselves. 

c. Benefits to the Navy . 

d. Data on reliability. 

e. Real or apparent drawbacks compared with 
earlier trainers. 

f. Real or apparent advantages compared with 
earlier trainers. 

g. Adjustments that must be made in the user's 
behavior patterns. 

h. How these adjustments will be achieved (formal 
retraining, assumption of persona 1 re sp on si- 
bi lity , et c . ) . 
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i. New responsibilities that are entailed. 

j. Present responsibilities that are to be 
reassigned to others . 

k. Who these other persons are and what 
preparation will be given for carrying 
them out. 

1. How those in the chain of command will 

be made aware that these responsibilities 
have been met. 

5. Who is to be in charge of promoting usage of the 
trainer (trainer advocate) . 

6 . What are the credentials and authority of the 
trainer advocate . 

7 . What threats the new device might present . 

a. PUys? cal . 

b. Threat to current confidence with respect 
to operating skills. 

c. Threat to other aspects of the user's self- 
image , prestige, etc. 

8. What the trainer advocate expects of the poten- 
tial users and what the users expect of him. 
Are the two sets of expectations exp licit and 
agreed upon by both parties? 

9. What provisions there are for direct experience 
with the new device. 

10. Whether the introduction communications (both 
oral and written) are designed for maximum 
appropriateness considering the characteristics 
of the users . 

A simple listing of factors,, however, is not enough to 
describe the process of acceptance or rejection. Clearly, 
this process is dynami c and many of the factors operate in 
an interactive way. An attempt has been made to summarize 
the factors hypothesized to influence training device ac- 
ceptance, and their interrelationships , in Figure 1 . It 
seems likely that a device will not be accepted or rejected 
in all details, but rather will have certain outstanding 
characteristics or deficiencies, whose combined influences 
place the device on some position on an acceptance-rejection 
scale. This consideration is reflected in the figure. 
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It was our objective to try to identify the influence 
of as many of the factors shown in Figure 1 as possible. 
We assumed from the outset that some of the factors in- 
fluencing acceptance would be subject to the control of 
design engineers and others would not. 

xNFLUENCE OF THE T.UINING SITUATION 

Perhaps the most important factors influencing training 
device acceptance insofar as the training situation is con- 
cerned are (1) the opportunity for training in the opera- 
tional environment (or lack of it), and (2) certain char- 
acteristics of the equipment or the environment that may make 
the regular practice of some skills virtually impossible from 
either a safety or economic viewpoint. If there is ample 
opportunity for practice of critical skills using operational 
equipment, the trainer is not likely to be the preferred 
training vehicle. However, there are nearly always some pro- 
cedures that, for practical reasons, simply cannot be sys- 
tematically trained during routine operations. Examples 
include drill in emergency procedures under simulated 
catastrophi c conditions, the firing of expensive or hi gh 
yield weapons, and the manipulation of environmental factors 
that have major influences on operating effectiveness, such 
as sonar detection ranges. 

Level and uniformity of skill development in the fleet 
is an obvious influence on the acceptance and use of train- 
ing devices. A less obvious influence is the amount of time 
actually available for training in relation to operating 
demands and administrative responsibilities that compete for 
the same time. This is a major consideration in evaluating 
statistical data that may reflect trainer acceptance such as 
pey*oent utilization figures. In many cases fleet personnel 
simply are not able to utilize a trainer to the extent they 
would like because of administrative duties. Any perceived 
deficiencies in the trainer, however, may serve to resolve 
the conflict between administrative demands and exercises in 
the trainer in favor of the former. 

MILITARY AND DESIGN SPECIFICATIONS 

It was beyond the scope of this study to perform a 
training situation analysis for all of the trainers whose 
acceptance was studied, or to analyze how fleet requirements 
and the inputs of educational specialists and engineers were 
translated, first into military characteristics, and then 
into training device design specifications. We felt it 
necessary, however, to examine the military characteristics 
for each trainer to identify its general design objectives. 
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Clearly, the question of trainer acceptance must be related 
to what it was intended to do and whether or not the users 
perceive it as fulfilling the same role as the designers did. 
It seems quite possible that a trainer might be accepted (or 
rejected) for a particular pattern of use which might be 
qualitatively different from the purposes stated in the 
military characteristics. 

SIMULATION FACTORS 

Prophet (14) has pointed out that a training device has 
one principal purpose--to provide an environment in which one 
can be trained. The most obvious characteristics of a 
trainer that influence its acceptance, then, are those fea- 
tures of the operating environment that have been selected 
for simulation and the fidelity of such simulation. Prophet 
has emphasized that simulators do not train; rather, it is 
the training program that trains. The critical factor is 
task fidelity • While we fundamentally agree with this view- 
point, it is nonetheless true that the features select ed for 
simulation determine important boundaries within which the 
training must be accomplished. In fact they determine the 
extent to which task fidelity is possible. We felt it was 
important therefore to identify all major are as of simula- 
tion applicable to major training devices and to determine 
user impressions of the adequacy of that simulation. The 
factors selected for study were as follows: 

SM-1. Simulation of the Internal Operating Environment . 
This factor was concerned with the adequacy of simula- 
tion of such characteristics of the internal operating 
environment as the type and location ot equipment, 
arrangement of displays and controls, illumination, 
noise, motion, temperature, vibration, and other in- 
ternal features that might be considered critical for 
training. 

SM-2. Simulation of the External Environment . This 
included consider at ion of such features as the visual 
scene, auditory stimuli or noise from the external 
environment , and ambient temperature , pressure, 
atmospheric or water conditions that might affect 
equipment or personnel performance. 

SM-3. Simulation of V eh i c le^ Equipment Performance. 
This factor was concerned with how well the per f ormance 
characteristics of the vehicle or equipment were 
simulated. In the case of vehicles, consideration was 
to be given to maneu ve r ab i 1 i ty , respons e t ime , and 
range of op er at ion; in the case of equipment, considera- 
tion centered on accuracy of operation, limits of 
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operational effectiveness, typical operating di f f iculties , 
etc . 

SM-4. Simulation of Controls . Control effects were to 
be considered from the standpoint of the "feel" of the 
simulated vehicle and the effects of control manipula- 
tions on how the system operated or how various in- 
formation was developed and displayed. 

SM-5. Simulation of Communication Procedures . This was 
concerned with the various communication links and pro- 
cedures in the trainer with respect to communications 
both within and between operational units. 

SM-6. Simulation of Communication Problems ^ Considera- 
tion was to be given to such problems as environmental 
noise that might interfere with communications, faulty 
equipment and various other sources of interference 
with the communicat ed information . 

SM-7. Simulation of Inf ormat ion Displays . This was 
concerned with the fidelity of information as displayed 
on scopes, plots, dials, and status boards. Both video 
and audio displays were to be considered. 

SM-8. Simulation of Sensor Performance . Consideration 
was to be givon to such factors as the physical repre- 
sentation of objects or conditions in the environment 
as sensed through the equipment; the number and variety 
of objects simulated ; how detectable the objects were 
as a function of environmental conditions, etc. If 
applicable, consideration was to be given to the dif- 
ferent classes of targets that were simulated. 

SM-9 . Target Evasion . For trainers that included 
target simulation , consideration was given to how 
realistically the target employed evasive tactics, 
agressive tactics , and count ermeasures . 

SM- 10 . Simulation of Weapon Firing and Control . For 
trainers that involved the firing or control of weapons, 
consideration was to be given to the simulation of tar- 
get tracking, localization, solution to the fire-control 
problem, choi^^e of weapons, weapon launching procedures, 
and realism of the results of the attack in terms of 
typical weapon effectiveness. 

TRAINING EFFECTIVENESS AND SITUATIONAL FACTORS 

Although the simulated operational environment provided 
by a training device is undoubtedly a .aajor consideration in 
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training device acceptance, it is by no means the only char- 
acteristic of major importance. In discussing the importance 
of training requirements information in the design and use of 
aviation training devices. Prophet {op, cit.) contended that 
the *'usuar* approach is to provide the ergineer with cockpit 
and performance specifications and let hia build the de^^'ice. 
Little mention may be made of what is to be learned, the 
psychological nature of the tasks involved. For Prophet, the 
most important characteristic of an effective trainer is "task 
fidelity.'* The achievement of task fidelity is made more 
likely by proper simulation but other factors must be taken 
into account. Further, the fidelity of simulation desired in 
a trainer is in part a function of situational factors in the 
operational environment . 

A number of situational and training requirements factors 
were hypothesized to be important to trainirg device accep- 
tance. These were as follows: 

TSF-1. Prob lem Setup . Within the purpose for which the 
trainer was designed, how readily can a variety of 
operational problems be set up on the trainer? Con- 
sideration was to be given to representativeness and 
scope of problems in relation to operational require- 
ments, and the relative ease or difficulty of setting 
up the problem for execution. 

TSF-2. Software . Consideration was to be given to the 
thoroughness, effectiveness, and sophistication of the 
utilization guides and other training materials asso- 
ciated with the trainer. The question was asked, 
"How effectively could the trainer be used if only the 
information contained in these guides were used to 
program and operate it?" (Technical and maintenance 
manuals were not to be included in this factor.) 

TSF-3. Comp leteness of Per form an ce Evaluation . Con- 
sideration was to be given to the trainer's provisions 
for providing in formation to individual s tudents con- 
cerning all important elements of their performance. 

TSF-4. Immediacy of Performance Evaluation . The 
trainer's provisions for immediate f eedb ack were to 
be considered in contrast to situations where there 
is feedback only after extensive delay. 

TSF-5. Reliability and Maintainability . Consideration 
was to be giv'^en to the amount of down time, and the 
level of effort required to keep the trainer operational. 
How readily can it be kept operational by the average 
training device technician? 
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TSF-6. Level of Training . Consideration was to be given 
to whether the trainer was suitable mainly for basic 
training and familiarization or whether it could be used 
effectively for advanced or refresher training as well. 

TSF-7. Training Opportunities . Consideration was given 
to the skills developed by the trainer in relation to 
the opportunity for acquiring and maintaining those same 
skills in the operational environment. 

TSF-8. Comprehensiveness of Training s Trainer capa- 
bilities were to be considered in relation to all of 
the skills that must be acquired for effective perfor- 
mance in a selected area of Navy operations. Are there 
any critical skills that receive insufficient emphasis? 

TSF-9. Value of Time in the Trainer. > In relation to 
all the activities that occur in the trainer, what 
proportion of time is spent in actually acquiring or 
improving essential skills. 

TSF-10. Use of the Trainer vs. Operational Equipment . 
One of the recognized advantages of trainers is that 
they permit training on activities that are either im- 
practical, too costly, or too d?,rgerous to perform on 
actual equipment. The disadvantage, however, is that 
the trainer may differ in som*) significant way from 
actual operational gear. The respondent was to consider 
how much advantage there is to using the trainer as 
configured vs. using actual operational equipment for 
training at shore -based schools . 

TSF-11. Repeated Use of the Trainer , When a trainer is 
used on some regular basis, at least some of the skills 
may already be well trained. Consideration was to be 
given to how useful the trainer is for repeated use in 
maintaining or refreshing the essential skills. 

TSF-12. Comparison with Similar Trainers . From an over- 
all training viewpoint, the respondent was to compare the 
trainer in sophistication and effectiveness with other 
, trainers that simulated similar Navy systems. 

UNDERSTANDING THE PURPOSES, CAPABILITIES, AND LIMITATIONS OF 
THE TRAINER 

The literature on the acceptance of innovations clearly 
reflects the dual roles of communications, either formal or 
informal, and direct personal experience with the innovation, 
in developing patterns of acceptance or rejection. We were 
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interested in three basic kinds of communications in relation 
to training devices and attempted, insofar as possible, to 
identify their existence and the possible roles played in 
generating each trainer's reputation . 

1. Formal or informal statements of trainer capa- 
bilities . 

2. Official directives concerning use of the trainer. 

3. The actions of individuals who were openly pro- 
moting the use of the trainer (trainer advocates). 

It vas assumed that direct personal use of a trainer 
would be a major factor governing level of acceptance. It 
is insufficient, however, simply to determine whether the 
trainer has been used by a particular individual, or how 
frequently he has used it, unless there is inquiry also into 
the manner of use. As suggested earlier, it is quite possible 
for a trainer to be used in one fashion, for example as a 
procedures trainer, when the military characteristics call 
for an operations trainer or a weapons system trainer. Con- 
sequently, an effort was made during the study to distinguish 
between acceptance of a trainer, as used, versus acceptance 
as designed. This included an inquiry into the qualifications 
of instructor personnel who operated the trainer and the level 
of satisfaction felt by the trainee as a result of his experi- 
ence in i t . 

CHARACTERISTICS OF THE USER 

The literature on attitude format i on suggests that many 
"psychological^* factors are involved in the acceptance of 
innovations' (Mackie and Mecherikoff , op. cit.) . Examples in- 
clude such individual characteristics as self-esteem, level 
of chronic anxiety, cognitive rigidity, need for authority 
approval, status in group, etc. Some of these factors are 
exttemely subtle and adequate techniques for their measure- 
ment probably do not yet exist. However, it was felt that 
some user characteristics could be measured reasonably objec- 
tively and an attempt was made to do so in the following 



1 . F elt level of personal competence in the technical 
area represented by the trainer . Except in the 
case of neophytes, each user of a training device 
has a different background of experience, compared 
to other Navy personnel, that possibly affects his 
attitude toward the trainer. 



areas : 
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2. Sufficiency of operational training . It was hy- 
pothesized that a major influence determing atti- 
tudes toward training devices was the extent to 
which the user felt t' at sufficient training was 
ob t ainab le in the ^ cal operat ional environment • 
Some Navy personnel seem to develop biases against 
trainers because of the assumption that the neces- 
sary operational skills can only be obtained in 
the operational environment • 

3 . Cognitive simplicity-complexity . This refers to 
the relative rigidity or flexibility with which an 
individual regards a particular innovation. Cog- 
nitive simplicity is exhibited by the tendency to 
categorize things as either all white or all black, 
or the inability to consider more than one dimension 
of an argument at a time. In the context of train- 
ing device acceptance, cognitive simplicity seemed 
to be reflected in the viewpoint of some that unless 
the trainer were perfect in ail respects, its use 
for any reason was a waste of personal time. 

4. Experience with other trainers of this type . Ex- 
perience with other trainers results in a set of 
personal expectations, often predominantly positive 
or negative, that was thought to condition acceptance 
of a new trainer. 

5 . Personal involvement with the design or original 
requirement for the trainer ^ The negative influence 
of the "not - invented here" phenomenon in developing 
attitudes toward innovations has been mentioned pre- 
viously. Personal participation in the development 
of an innovation and the opportunity to contribute 
to the solution of a problem in one's own area of 
expertise plays a highly significant role in accep- 
tance. The great majority of Navy personnel who use 
training devices have^ of course, no opportunity to 
contribute to the design of that device. Neverthe- 
less, they are continually passing judgment on how 
well someone else designed the device. 

ACCEPTANCE DETERMINED BY CONGRUENCE BETWEEN PERSONAL NEEDS 
AND TRAINER FEATURES 

At the final stage in the dynamic process depicted in 
Figure 1, it was hypothesized that the level of acceptance 
for any particular trainer was a complex function of the felt 
needs of the individual (both technical and psychological) , 
and the perceived value and deficiencies of the trainer 
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resulting from various types of communications and direct per- 
sonal experience. The word perceived is emphasized because 
we did not assume that the judged characteristics, value, or 
deficiencies of the trainer would necessarily be based on 
objective fact. Further, there was reason to believe that 
most users lacked detailed knowledge of the design objectives 
and military characteristics of the trainers. 

There was, of course, no direct measure of congruence. 
However, it was possible to obtain measures that apparently 
reflected deficiencies in trainer characteristics in relation 
to the felt importance of those characteristics for effective 
training. The methodology employed is described in the 
foil owing section. 
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SECTION III 
STUDY METHODOLOGY 

To summarize the considerations set forth in the previous 
section, this study was particularly concerned with five major 
sources of influence on the process of trainer acceptance: 

1. The training situation , particularly the level and 
uni f ormity of ski 1 1 deve lopment in the fleet; the 
opportunities for training certain functions in 

the operational environment; various characteristics 
of the equipment and environment that influence the 
opportunity to exercise particular skills; and the 
training time available in contrast to operating and 
administrative demands . 

2. Trainer characteristics . Two broad areas were 
considered : (a) simulation factors; (b) training 
effectiveness and situational factors, including 
convenience, location, availability, and reliability. 

3. Understanding of the trainer . The user's comprehen- 
sion of the trainer's capabilities, limitations, and 
purpose; knowledge of the trainer through both formal 
and informal communications and through direct per- 
sonal use. 

4. User characteris t i cs . The user's felt level of con- 
fidence in the technical area represented by the 
trainer; his experie^ e with other similar trainers; 
his personal invo 1 vem^iit with the design or suc- 
cessful use of the trainer; his assessment of the 
training need. 

5. Congruence . The degree of conformity (or disparity) 
between (a) the physical characteristics of the 
trainer; (b) the user's unders t anding of its capa- 
bilities and limitations; and (c) how important he 
judged these various characteristics to be for 
effective training. 

The investigation required the development of a means to 
obtain scaled judgments of all trainer and user character- 
istics hypothesized to exert major influences on the accep- 
tance process. It was desired to develop definitive profiles 
of different trainers that, on tne basis of various opera- 
tional criteria, could be shown to be differentially "accepted" 
or "rejected," "effective'* or "ineffective." 
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To accomplish these objectives, the following procedures 
were followed: 

1. Specially designed quantitative rating scales were 
developed that could be used to rate training de- 
vices on 10 areas of simulation and 12 training 
effectiveness and situational factors. 

2. A structured questionnaire was developed to provide 
detailed information that might amplify the ratings. 

3. A group of training devices was identified whose 
members were thought to reflect significant dif- 
ferences in level of acceptance as well as various 
kinds of operational training. 

4. Qualified gvoups of users were identified, con- 
tacted and interviewed using the rating scales and 
structured questionnaires . To insure that various 
viewpoints were adequately represented, the user 
sample was carefully selected from administrative, 
instructor, student, and maintenance personnel 
associated with each trainer. 

5. An operational definition of acceptance (rejection) 
had to be developed that reflected, in some logical 
way, all avail able objective and subjective data. 
In addition to opinion data, it was considered 
desirable to include objective indices such as per- 
cent utilization and the percent of requested 
training accomp 1 i shed . * 

6. A technique had to be developed whereby some of the 
more subtle factors, such as felt level of com- 
petence, and the perceived import an ce of various 
characteristics of the trainer could be conveniently 
ident i fied and quantified. 

SELECTION OF TRAINING DEVICES FOR INCLUSION IN THE STUDY 

The selection of devices for inclusion in the study was 
a critical step in the investigation. On the one hand, the 
sample size had to be large enough that one could feel con- 
fident about generalizing the results to the population of 
trainers. Clearly, this objective could not be reached if 
only two or three trainers were rather intensively studied. 



★ 

This was defined as the number of hours used divided by the 
number of hours requested. 
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The possibility existed that there might be different con- 
siderations with respect to comparatively simple part-task 
trainers from those pertaining to very complex weapons sys- 
tems trainers. Another issue was whether trainers designed 
for air operations would involve the same kinds of acceptance 
criteria as trainers oriented toward surface ship or sub- 
marine training. 

For the study to be of maximum value, it was decided 
first that the sample should include trainers that were rela- 
tively complex and represented comparative ly large investments 
either in ter^ns of their size or the number of installations 
throughout the Naval establishment. Second, it was considered 
necessary to include trainers that represented air, surface, 
and subsurface operations . Third, a decision was made to 
include trainers which were known, or suspected, on the basis 
of a priori information, to represent cases of both high and 
questionable acceptance. (In one case there was evidence of 
outright rejection.) Finally, it was decided that a few 
trainers should be included where the installations at two 
different locations represented certain known design differ- 
ences. This would make it possible to determine whether the 
measurement techniques were sensitive to these differences. 

It was not possible, however, to establish level of ac- 
ceptance on any thorough bas is prior to the investigation . 
For the most part, the only a priori information available 
to the investi gators were the general impress ions of educa- 
tional specialists and engineers at NTDC together with some 
available data on the extent of utilization. The latter 
criterion was recognized as an imperfect index of acceptance 
because of the many practical considerations that influence 
trainer utilization that have little to do with acceptance. 
It was decided, therefore, that a posteriori criteria would 
have to be developed from the study data to help identify 
'•effective" and "ineffective" trainers. In addition to the 
criteria already mentioned, these would include the method 
in which the trainer was actually being used (i.e., in rela- 
tion to its design objectives) and the various expressions 
of user attitudes toward the value of time spent in the 
trainer . 

The sample of 16 training devices selected for the study 
is listed in Table 1. It will be evident that the trainers 
were very heterogeneous in nature, involving surface, sub- 
surface, and airborne operations, and including devices in- 
tended for part-task training of individual operators up 
through some of the most complex coordinated tactics trainers 
ever deve loped . 
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TABLE 1. SAMPLE OF TRAINING DEVICES 



Uc s 1 y n a t or 


_ 

Type 


Locari on 
(of Unit btudied) 


Weapons 

Q c ^ o m c 

Trainers 2F66A 


S-2E a 1 rcrew 


North Island 


9 F ft ft A 


O'-ZL aircrew 


yuonseT ro i nt 


2F55B 


F-4B a i rcrew 


M i rama r 


2F65 


E-7.B a i rcrew 


North Island 


2F69A 


P-3 a i rcrew 


Barber ' s Point 


2F69B 


P-3 a i rcrew 


Moffett Field 


2 1 A38 


SS(N) S/M crew 


Pear 1 Harbor 








Flight 

Trainers 2F64B 


SH-3D he 1 i copter 


Quonset Po i n t 


2B2 1 


T-28 i nstrument 
t ra i ner 


Pensaco 1 a 


2H87 


Carr i er approach 
1 and i ng 


Pensaco 1 a 








Tacti cal 
Team 

Trainers 1 4A2 


ASW team t ra i ner 


San Diego 


1 4A6A 


Coord i nated ASW 
tact i cs 


San Diego 








oc n s 0 r 

U)JCl aCUl 

Trai ners 

1 4B35 


JULIE/JEZ 
operations 


Moffett Field 


1 4B40 


Radar/Mad 
operat ions 


Pax River 


1 5EI6 


E 1 ect ron i c 
countermeasures 


Moffett Field 








Emergency 
Procedures 
Trai ner 

2 IC5 


Submarine casualty 
contro 1 


Pear 1 Harbor 
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PROCEDURE 

The basic procedure followed during the study was for 
one or more members of the HFR staff to visit a trainer 
installation, usually for a period of several days • During 
this time, project personnel became acquainted with the 
trainer, how it was being used, and the instructor and main- 
tenance personnel assigned to it. Arrangements were made 
to administer the rating scales and questionnaires to key 
administrative, instructor, and maintenance personnel and to 
a representative group of students. 

The rating scale and questionnaire were designed so that 
they could be self-administered. However, it was felt that 
the resulting data would be much more reliable if each in- 
dividual participant was personal ly instructed by a project 
member in the objectives of the study, and the mechanics of 
completing the rating scale and questionnaire.* Wherever 
possible this was done with groups of individuals and the 
participants were encouraged to complete the data forms, in- 
dependently, asking questions of the project representatives 
as they arose. In some cases it was necessary to permit the 
respondent to complete the data forms at their convenience 
and later return them to the project representative at the 
trainer site. On the average, about twenty-five minutes was 
required to complete the rating scale and thirty-five or 
forty minutes for the questionnaire. 

Upon receiving the completed materials, the project 
represent at ive immediately reviewed all ratings and responses 
to various key questions (such as statements about defi- 
ciencies in the trainer, recommendations for changes or 
improvements, etc.) for the purpose of eliciting additional 
detailed information about attitudes, both positive and 
negative, toward the trainer. In a few cases, these inter- 
views were short and perfunctory ; in many cases, however, 
they generated extended commentary by the participants in 
discussions that sometimes lasted an hour or more in length. 
The time required for complete data collection at a trainer 
site, involving on the average about twenty participants, 
varied from as little as two or three days to as long as a 
week depending on the availability of personnel. For ex- 
ample, in some of the complex team trainers , the trainees 
could not be interviewed at the trainei* site but rather had 
to be located on individual ships at a time when their usually 
busy work schedules permitted an opportunity for discussion. 



Examples of the Rating Scale and Questionnaires are given 
in Appendix A. 
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A description of the 326 part icip ants in the study is 
contained in Table 2, They varied greatly in rank and in 
personal experience in the area of operations represented 
by the trainer. They were classified as having low, medium, 
or high experience in accordance with the distribution of 
self-ratings of experience shown in Table 2, 

Most had had considerable experience with the trainer 
they were called upon to rate. Fifty-five percent of the 
respondents knew the trainer in the role of student; the 
remaining 45% was comprised of instructors, maintenance 
personnel, and administrators. The number of people shown 
in Part C of Table 2 exceeds the number of trainees shown 
in Part A because some administrative and instructor person- 
nel had also used the trainer in the role of student. 

For each trainer that was designed for team training 
an attempt was made to obtain opinions from four levels of 
pers onne 1 : 

1, Officers with command responsibility, 

2, Officers having responsibility for technical 
specialties (for example, CIC officers, ASW 
officers, TACCO^S, engineering officers, etc), 

3, Senior or supervising petty officers, 

4, Basic operators of major units of equipment. 

With respect to non-students, an attempt was made to 
obtain the opinions of the following types : 

1, Administrative personnel (the officer in charge 
of the training facility, the operational staff 
training officer, the school training officer, 
etc , ) , 

2, Instructor personnel. The principal instructors 
for all stations, both officer and enlisted, 

3, Maintenance personnel (the electronics material 
officer having responsibility for the device, 
as well as the leading enlisted maintenance 
personnel). In some cases these were Training 
Devicemen; in others, they were ET ' s , SCJ's, 
OS's, etc. 
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Since it was hypothesized that acceptance of a training 
device might be a function of how expert the participants 
considered themselves to be in the technical area represented 
by the trainer, all participants in the study were asked to 
rate themselves on a O-to-15 point scale with respect to the 
following statement: 

"Compared with other Navy personnel who work 
in the area of operat i ons or type of equip- 
ment represented by this trainer, how 
extensive is your experience? Consider the 
number of closely related billets you have 
had and recency of experience." 

The distribution of ratings on this variable are shown 
in Part D of Table 2. It seems evident from the wide dis- 
tribution of scores that the participants were quite candid 
in reporting their experience. To interpret this result, 
the following definitions of anchor points which appeared 
on the scale will be helpful. 

0 - My experience in this area of operations or with 
this equipment is nil. 

5 - My experience is somewhat limited; the majority 
of my assignments have beev^ in other areas or 
with other equipments . 

10 - My experience is substantial; although I have 

had other types of assignments, I have had more 
in this area of operations, or with this type 
of equipment than any other. 

15 - My experience is extensive; I consider this 

operational area, or particular type of equip- 
ment, my special fie Id. 

For the purpose of some data analyses, the participants 
were divided into three experience groups at the points shown 
in the Table. Those classified as ''low" all rated themselves 
below the midpoint of the scale, the majority describing them- 
selves as being somewhat limited in experience with this type 
of equipment. A large middle group, who rated themselves from 
just above the midpoint of the scale to about the three-quarter 
point, were classified as being of •'medium" experience. Fin- 
ally, a third group that rated themselvef in the upper one- 
fourth of the scale were classified as "high" in experience. 
While this method of classification was somewhat arbitrary, 
it was felt to be adequate for testing the general hypothesis 
that level of experience is an influencing variable insofar 
as trainer acceptance is concerned. 
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SECTION IV 

DEVELOPMENT OF AN OPERATIONAL DEFINITION 
OF TRAINER ACCEPTANCE 

NATURE OF THE STUDY DATA 

The basic data deriving from the study were in the 
following forms : 

1. Numerical ratings o.' the judged adequacy of each 
trainer on each of 10 simulation factors. 

2. Numerical ratings of the judged importance of 
each s imu 1 at i on factor • 

3. Numerical ratings of 12 characteristics relating 
to training effectiveness and situational factors. 

4. Responses to the questionnaires designed to 
amplify the information obtained from the 
rating scales . 

5. Data on trainer utilization from the NTDC trainer 
utilization reports . 

With respect to the rating scale data, a numerical score 
was assigned (from 0-15) representing the judgment of each 
participant on each characteristic of uach trainer. This was 
a simple one-to-one linear translation of the location of the 
check mark in relation to the numerical indices on the scale 
(see Appendix A). These values were averaged across all 
respondents . 

With respect to questionnaire data , frequency counts 
were made of the various categories of response provided. In 
the case of open-end questions, the nature of the response 
was coded in terms of its substantive content, and whether 
it was a generally positive endorsement of the trainer or 
reflected a negative viewpoint toward it. It was determined 
for various characteristics of each trainer whether the pre- 
ponderance of comments made was positive or negative and, 
further, whether the distribution of positive or negative 
reactions was disproportionate in relation to the comments 
offered on all trainers as a group. 

Both the rating scale and questionnaire data represented 
the subjective opinions of the users, although every effort 
had been made through the design of the data collection in- 
struments to structure the judgmental task as carefully as 
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possible. It was considered desirable, however, to include 
at least one source of external data, preferably totally 
objective in form, to complement the opinion data in forming 
a definition of acceptance. This requirement was met by the 
inclusion of utilization data from the Training Device 
Utilization and Application Report (NAVTRADEVCEN Report 
10 171-4), which provided month ly data on utilization of all 
training devices. Utilization data do not, in themselves, 
constitute an adequate criterion of trainer acceptance. 
Never the less, they have the merit of a high degree of ob- 
jectivity and logically cannot be excluded from a compre- 
hensive definition of acceptance. Some of their deficiencies 
are described later. 

A COMPOSITE INDEX OF ACCEPTANCE 

In the interest of convenience and to facilitate com- 
munication, it was desirable to develop a single composite 
index that reflected, as comprehensively as possible, the 
obtained data about each trainer that apparently reflected 
its acceptance by the users. .Although many approaches to 
such an index were possible, it was felt that the index 
should reflect the user's ratings concerning the value and 
comprehensiveness of training; their judgments concerning 
the adequacy of simulation; their voluntary comments about 
the trainer's particular capabilities and limitations as 
reflected in the questionnaire; and the objective data 
concerning actual utilization. 

A composite ''acceptance" score was developed from six 
subscores that in turn were defined as follows: 

1. Felt value of training . This score was a simple 
summation of the rating scale values assigned to 
the trainer by all respondents on the following 
five scale)s from the training effectiveness 



s e c t ion . 

a . Compr ehensivene s s of training. 

b. Value of time in the trainer. 

c. Use of the trainer vs. operational equipment. 

d. Value of repeated use of the trainer. 

e. Comparison with similar trainers. 



The resulting scores for all trainers are shown in Column 1 
of Tab le 3 . 
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Simulation deficiencies. Since each training device 
was rated not only on the simulation factors per se 
but also on the felt importance of each simulation 
factor, it was possible to compare each rater's 
judgment of the importance of each factor to the 
score he assigned his particular trainer on that 
factor. If the judgment of importance were higher 
than the score assigned to the trainer, this re- 
sulted in a "negative differential" score for that 
trainer on that factor. Negative scores were inter- 
preted to mean that in the view of the user, the 
trainer had a deficiency in that area of simulation. 
Two kinds of deficiency indices were calculated for 
each trainer : 

a. The algebraic sum of differential scores on all 
10 simulation factors; 

b. The number of simulation factors on which each 
trainer received a deficiency score of 1 rating 
scale unit or more. 

It was hoped that both the severity of the felt 
deficiencies and the generality of the deficiencies 
would be reflected in the two scores. The resulting 
scores are shown in Columns 2 and 3 of Table 3. 

Voluntary positive and negative statements about 
each trainer . There were seven open-end questions 
in the questionnaire which provided the respondents 
with an opportunity to reflect favorable or un- 
favorable judgments about the trainer. 

Attention was focused on these particular questions, 
not only because of their content, but because some 
effort on the respondent *s part was required to 
formulate his answer. For this reason it was felt 
that these responses might be particularly relevant 
to underlying user attitudes. 

Responses to the following quest ions were classified 
as essentially positive or negative and then tallied 
for each trainer : 

a. Based on your experience with the trainer, v»hat 
would you be likely to say to someone else about 
it, both pro and con? 
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b. As far as you know, does this trainer have any 
kind of a reputation with regard to maintenance 
or reliability? For instance, is it regarded 
as a generally reliable or generally unreliable 
machine? I would like to know if the trainer has 
a reputation in this respect, even if it might 

be different from your own opinion, 

c . To your knowledge , are there any people now, or 
have there been any in the past, with an un- 
favorable opinion of the trainer? 

d. Considering your training needs and the needs of 
other trainees, what limitations are there in 
what this device was designed to do? 

e. Overall, in terms of actually accomplishing ef- 
fective training, what would you say are the 
most serious deficiencies of trainers of this 
type? 

f. What changes would you suggest in order to make 
it easier or more pleasant to use the trainer 
without sacrificing necessary realism? 

g. What design changes would you suggest to make 
the trainer more cost-effective? 

For each trainer the relative frequencies of positive 
and negative statements to each question were tabu- 
lated. Then, it was determined whether the distribu- 
tion of positive and negative statements about that 
trainer differed substantially from the distribution 
for all trainers as a group. In cases where the 
trainer received a disproportionate number of posi- 
tive or favorable comments, the trainer was given a 
score of 2 for that question. In cases where a dis- 
proportionate number of un favor able responses was 
recorded for a particular trainer, that trainer re- 
ceived a 0 for that particular question. In the 
event that the responses were generally positive but 
not disproportionate to those for the total popula- 
tion of trainers, a score of 1 was awarded. 

A total score reflecting the reactions of all re- 
spondents to all seven questions was then developed 
for each trainer. The results are shown in Column 
4 of Table 3. 
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Trainer utilization data > As was suggested earlier, 
the amount of utilization of a trainer may or may not 
be an indication of its acceptance by the users. 
Many administrative and practical constraints affect 
the extent to which a trainer is utilized. A trainer 
that is held in high regard by operational personnel 
may simply not be scheduled because of the urgency of 
upkeep responsibilities. Conversely, a trainer can 
be heavily utilized without necessarily being highly 
accepted by its users because it is the only thing 
available. For the most part, however, high utiliza- 
tion probably signifies substantial acceptance. For 
this reason it was felt that utilization data should 
comprise a part of the composite index of acceptance. 

Data on utilization were taken from a four-month 
sample of NTDC's records. The data were selected 
from the four -month reporting period immediate ly 
prior to the time that this study went into the 
field. In most cases, only dat. covering the par- 
ticular trainer installation that the project team 
visited were utilized. In one or two cases, data on 
the same trainer at several different locations had 
to be substituted because there was no report from 
the location studied. In one case, the trainer in- 
stallation was too recent for a data base to have been 
built up. Two measures of utilization were used: 

a. Percent utilization . This was defined as the 
total number of hours per month the device was 
used for student training, regardless of type, 
divided by 160 hours (the number of hours pre- 
sumed to repres ent full availability over a 
working month) . These scores could exceed 100% 
if the device was typically used on more than 
one shift. (See Column 5 , Tab le 3 . ) 

b. Percent of requested training accomplished . Be- 
cause of the shortcomings associated with a 
straight percent utilization factor, it was decid- 
ed to include a second index from the training 
device utilization report. This was based on the 
ratio of the total hours the device was used for 
training to the total number of training hours 
requested. It was felt that this index might pro- 
vide an additional indication of the user's regard 
for the trainer since a trainer with a compara- 
tively low percentage utilization might neverthe- 
less be utilized a high percentage of the time 
the users had available for training. Scores on 
this variable are shown in Column 6 of Table 3. 
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The next step in developing the composite index of ac- 
ceptance was to combine the scores for the six separate 
elements described above. In the absence of any obvious 
rationale for doing otherwise, it was decided that a simple 
unweighted composite of the six scores would serve as the 
desired criterion. However, since the raw scores had 
greatly different means and variances, it was necessary 
first to convert these to standard scores as shown in Table 
4. The algebraic sum of the standard scores was then taken 
as the composite index. 

One other procedure needs to be noted. The size of the 
scores on Elements 2 and 3, in their raw form, reflected 
negative viewpoints toward the trainer's simulation charac- 
teristics whereas in all other cases, the higher the score 
the more positive the attitudes presumably were. Conse- 
quently, in computing the sum of standard scores, the signs 
for these two variables were reversed. 

Table 4 shows that the composite criterion scores ranged^ 
from +6.36 to -10.11. The mean was 0.7. For the purpose of 
definition, trainers with standard scores of +3.0 or better 
were classified as generally highly accepted; those with -3.0 
or less were regarded as having serious acceptance problems. 
Table 4 is useful for identifying the reasons for the mag- 
nitude of the composite scores. The trainers are ordered 
in accordance with their score on the first element of the 
composite criterion, i.e., the summation of the five train- 
ing value ratings assigned by the users. 

In cases where similar trainers are involved, the source 
of any difference in their composite scores can be determined 
by inspection. In comparing the 2F66A's at San Diego and 
Quonset Point, for example, it is evident that the former 
trainer received a somewhat higher overall criterion score 
than the latter. Inspection of the various elements of the 
criterion reveals notable differences between the trainers in 
judged adequacy of simulation and quite different patterns of 
utilization. However, they received very similar scores with 
respect to positive recommendations (Element 4) . 

The 2F69A and 2F69B offer another such comparison. In 
this case, the 2F69B received a considerably higher composite 
score than the 2F69 A. However , it is evident that most of 
this difference stems from differences in utilization. The 
2F69B, in fact, received a lower overall rating on effective- 
ness factors, although it did receive a somewhat higher rating 
on adequacy of simulation than did the 2F69A. 

The significance of these scores in relation to the over- 
all acceptance profile for each trainer is discussed in greater 
detail in Section V. 
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INTERCORRELATIONS OF THE ELEMENTS IN THE COMPOSITE CRITERION 
OF ACCEPTANCE 

Since the various elements making up the compos it e cri- 
terion reflected a mixture of relatively subjective and 
objective measures, it was of interest to determine the ex- 
tent to which they correlated with one another. In fact, 
the utilization data could be considered as external cri- 
teria against which the rating and questionnaire data might 
be validated. Large correlations probably would not be 
expected, however, because of the previously mentioned fal- 
lible nature of the utilization data. The int ercorre lat i ons 
of the 6 elements are shown in Table 5. 



TABLE 5. INTERCORRELATIONS OF SIX ELEMENTS 
OF THE COMPOSITE CRITERION OF ACCEPTANCE 





Element 
1 


Element 
2 


Element 
3 


Element 
4 


Element 
5 


Element 
6 


Element 1 
Training Value 


1 .00 


-.72 


-•28 


• 73 


• 12 


• 54 


Element 2 
Simulation 
Deficiencies 
(Alqebralc) 


.-•72 


1 •GO 


.60 


-•70 


-.54 


-•7 1 


Element 3 
Simulation 
Deficiencies 
(Number) 


-•28 


• 60 


1 ,00 


-•58 


-.30 


-•23 


Element 4 

Positive 

Recommendations 


.73 


-•70 


-.58 


1 •OO 


• 10 


• 25 


Element 5 

Percent 

Utilization 


. i2 


-.54 


-•30 


• 10 


1 •OO 


• 61 


Element 6 
Hours Used/ 
Hours Requested 


• 54 


-•71 


-.23 


.25 


• 6) 


I •OO 



It is evident that the various elements derived from the 
rating scale and questionnaire data correlated substantially. 
This was to be expected because they reflected a common source 
of opinion- -name ly that of the users^ It is notable, however, 
that Element 1 (Training Value) correlated .54 with Element 6 
(Hours Used/Hours Requested) and that Element 2 (Simulation 
Deficiencies) correlated -.54 with Element 5 (Percent Utili- 
zation). These are substantial re lat ion ships between very 
independently derived measures and strengthen the conviction 
that the various indices do reflect underlying attitudes of 
acceptance • 
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On the other hand, none of the correlations was so high 
to suggest that any of the elements was simply duplicating 
the contribution of another in making up the composite cri- 
terion. The highest correlation (,73) was between the judg- 
ment of Training Value, Element 1, and the number of dis- 
proportionately positive statements volunteered about a 
trainer (Element 4), The two scores reflecting utilization 
(Elements 5 and 6) correlated only ,61, indicating that 
they do, to some extent, mean different things with respect 
to the pattern of use. 

The composite criterion score was used to classify the 
trainers in the study into three broad categories : 

1. High acceptance. 

2. Moderate acceptance. 

3. Low acceptance. 

In the section that follows, the individual trainers 
comprising these groups are examined in detail and further 
interpretation is made of the probable significance of the 
composite criterion score, and its elements, for each 
trainer individual ly. 
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SECTION V 

ACCEPTANCE PROFILES OF INDIVIDUAL TRAINERS 

The main results of the study have been summarized in 
descriptive profiles for each trainer. Two types of profiles 
were prepared, one reflecting the 10 simulation factors and 
the other the 12 situational and training effectiveness 
factors . 

The profiles permit easy analysis of the judged capabil- 
ities and limitations of each trainer and how each trainer 
compared with all others in the study. In addition, they 
reflect how important the participants considered a parti- 
cular feature to be and whether the trainer measured up to 
their expectations on each factor. 

INTERPRETATION OF THE PROFILE 

The profile on simulation factors is shown in two di- 
mensions, one boundary reflecting the average ratings given 
to the trainer on each factor and the other reflecting the 
judged importance of that factor for effective training. 
In Figure 2, a composite simulation profile for all trainers 
is shown. The heavy dark line shows the average values 
assigned to the trainers; the shaded area represents the 
amount of difference between those values and the judged im- 
portance of each factor (importance differentials) . Thus 
the amount of shaded area can be interpreted rather directly 
as an indication of the amount of felt deficiencies. It is 
clear, from Figure 2, that the trainers as a group were 
felt to be most deficient in simulation of vehicle performance, 
effects of controls , displayed information , sensor performance , 
and target reactions. They were generally in accord with ex- 
pectations in simulation of internal and external operating 
environments , communi cation procedures , and communi cation 
prob lems . 

It will be noted that, in general, the profile for the 
group of trainers varies about a scale value of approximately 
10. In designing the rating scales (Appendix A) a scale 
value of 10 was defined as generally adequate or accept able. 
Deficiencies might be recognized but they were not considered 
so serious as to detract significantly from the value of 
trainer experience. Scale values of 5 or lower were defined 
as an indication of an unacceptable feature or condition. 
Finally, scale values of 15 were defined as reflecting 
clearly superior characteristics , perhaps (in the case of 
simulation factors) being virtually non-discriminab le from 
operating equipment or conditions. 
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Both the scale values assigned and the importance dif- 
ferentials should be taken into account in interpreting the 
profiles. For example, it is clear from Figure 2 that most 
trainers were rated low in simulation of communication prob- 
lems; however, the importance differential for this factor 
was quite small. Most users did not consider this to be a 
serious deficiency . 

The composite profile for all trainers on training ef- 
fectiveness and situational factors is shown in Figure 3. 
It is evident that, as a group, the trainers were considered 
most deficient in the areas of software and completeness of 
performance evaluation. They received their highest assess- 
ments in the opportunity they provided for supplementing 
operational experience and value of repeated use (refresher 
training) . 

The profiles of individual trainers show much greater 
variations than the two "average" profiles presented in 
Figures 2 and 3. On the following pages, profiles are shown 
first for a group of highly "accepted" trainers, then for a 
group that clearly have acceptance problems. Finally, 
trainers that are more or less average in acceptance are 
described. The criterion of acceptance was, of course, the 
composite standard score developed for each trainer and shown 
in Table 4. 
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PROFILES OF SOME HIGHLY ACCEPTED TRAINERS 

DEVICE 21C5: ADVANCE SUBMERGED CONTROL TRAINER, The 21CS 
was designed to be a "shore-based, realistic, high fidelity, 
dynamic training device capable of providing training for 
problems peculiar to steering, diving, and casualty control 
operations in STURGEON class submarines It was intended 
to provide training in the principles of diving, surfacing 
and submerged control; casualty and emergency operations that 
might occur singly or in combinations; to demonstrate the 
effects on ships' performance of various casualties; and to 
accurately simul at e critical recovery factors and effects 
such as time required to stop flooding, the action of planes 
and rudders, speed, main ballast tank blow and vent, negative 
tank blow and vent, and low pressure blow system. The de- 
vice is mounted on a platform which is free to move in pitch 
and roll. Movement in depth is simulated through sensor 
displays . 

The device is computer controlled in accordance with the 
latest available equations of motion for the STURGEON class 
submarine (SSN637) . The simulated effects of environment 
include surface effects, wave motion (various sea states), 
and bathythermal effects. The steering and diving stations 
are faithful reproductions of the submarine's interior. 
The platform accommodates three planesmen, one ballast control 
panel operator, and one diving officer. Simulated malfunc- 
tions and flooding are controlled by the instructor by means 
of a keyboard control. 

The 21CS is a highly accepted trainer with a composite 
criterion score of +3.5 (see Table 4). The acceptance pro- 
files for the 21CS are shown in Figures 4 and 5. With 
respect to simulation, it is clear that the design objective 
of high fidelity simulation was clearly met in the eyes of 
the fleet. The device was rated uniformly higher than 
average in virtually every respect; further, the discrepan- 
ancies between felt importance of various simulation factors 
and their actual implementation in the trainer were, in 
almost every case, very small. Particularly high ratings 
were obtained with respect to simulation of the internal 
operating environment, vehicle performance, effects of con- 
trols, information displays, and sensor performance. 

With respect to training effectiveness factors , the 
21C5 also shows a highly superior profile. It was rated 
particularly high in problem setup, software, reliability, 
level of training, the training opportunities it provides, 
value of repeated use, and in overall comparison with similar 
trainers. Only in completeness of performance evaluation did 
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it fall to approximately the average level assigned to all 
trainers. This is an interesting exception to the generally 
very high rating of this trainer. It may reflect a concern 
expressed at the training facility over the lack of perma- 
nently assigned instructor personnel who are qualified in sub- 
marine operations. The of f icer~in-charge indicated that the 
availability of a qualified officer for this billet would 
greatly enhance the trainer's value. The instructor operating 
the trainer at the time of our visit was a highly motivated 
training deviceman who had "never seen a submarine." He had 
essentially taught himself how to instruct others in the use 
of the trainer through a period of on-the-job training. Un- 
doubtedly he was greatly aided in this process by the Instruo-- 
tors Handbook for Advanced Submerged Control Trainings-Device 
21C5 prepared by the Electric Boat Division of General Dyna- 
mics. This publication provides 25 diagnostic exercises with 
detailed descriptions of the training objectives, the respon- 
sibilities of the instructor, and the means of problem setup. 
It is interesting to note that this publication strongly rec- 
ommends that the instructor be a qualified GOD on a nuclear 
powered submarine or a qualified engineering watch officer. 

The point was emphasized by some submariners that it is 
desirable to use their own personnel as instructors because 
of the need to follow their own particular standing orders. 
They felt the instructors at the device knew the basic opera- 
tion of the trainer well enough, but not their own particular 
operating procedures. 

"The instructor should be a qualified submariner so 
that the proper information is given to the crews 
at the proper time and in proper sequence. As far 
as operation of the trainer , the instructors are 
more than qualified. As far as instructing the cor- 
rect procedures, there are times when it takes a 
qualified submariner to know what the causes and 
effects are . " 

Th , instructors themselves strongly felt their own in- 
adequacies : 

"The operators are well-qualified to operate the 
trainer, but have never been trained in operating 
actual equipment. I don't see how an inexperienced 
person can instruct to the full capabilities of 
the trainer. I have seen a fully qualified man 
instruct and he can cover so much more it's ridic- 
ulous . " 

It was felt by training personnel that acceptance of 
the trainer would vary among individual submarine crews. 
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Utilization had increased since notices were sent out from 
the training center describing the trainer's capabilities . 
However , it was felt that some submarine crews would not 
use the trainer because it did not represent their esact 
configuration . 

It was evident that most users accepted the trainer 
highly, even though they detected certain ways in which it 
was different from actual equipment. One planesman com- 
mented : 

"It reacts a little faster than our boat--but every 
plane feels different. It's just a matter of 
getting used to the feel." 

"The trainer is dry. The boat is not. You don't 
get some of the environmental effects (conse- 
quences )- -you • re not under the same pressure." 

"The trainer acts a little different from the boat. 
Also, the gauges are set up somewhat differently. 
Some of the indications are instantaneous , whereas 
in the boat the changes are gradual." 

"Background sounds are not as high as they are on 
the boat- -it lacks realism in this respect. You 
notice every sound when you have a casualty. For 
example, the blow valve is extremely loud. We're 
training mostly new people--it's OK for them. If 
it were used for refresher training, some of the 
noises would be desirable. I couldn't talk to you 
if we had an actual casualty because of the back- 
ground noise ." 

It was stated that the trainer originally had been pro- 
grammed differently than it is now and was not acceptable 
until it was programmed to more closely reflect the per- 
formance of the 637 class submarine. 

"Originally the program was designed toward what 
they 'thought' the 637 would do on the basis of 
theory. It was necessary to change the program 
to reflect the operating characteristics that the 
submarine proved to have with experience. There 
were many initial complaints when the trainer 
was first installed for this reason. Operational 
feel and handling are very specific to a submarine 
class and may prove to be different from design 
. expectations • " 
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In contrast to this view was the following comment: 

'•The trainer is very good, even for SSN594 class 
submarines. The control room arrangement is 
different but the response characteristics, which 
is the important thing when training on ship's 
control, is very similar and should prove to be 
very useful for training.'' 

However, the same individual, when asked how he would possibly 
like to see the trainer improved, could suggest only that the 
trainer be made specifically for the SSN595 class. 

It will be recalled that the participants in the study 
were divided into three groups based on their level of ex- 
perience in the area of operations represented by the trainer. 
The ratings of each trainer were analyzed separately for each 
experience group to determine whether there was differential 
acceptance as a function of felt "expertise." The results 
for the 21C5 are shown in Table 6. It is evident that there 
is no clear relationship and that acceptance was high for all 
groups. For reasons that are not clear, the "medium" ex- 
perience group rated the trainer lower than the other groups 
on simulation of communications, communication problems, 
sensor performance and external operating environment. Per- 
haps this group is at a level of training that makes one 
particularly sensitive to any detected deficiency. 

TABLE 6. AVERAGE RATINGS OF THE 21C5 
BY LEVEL OF OPERATIONAL EXPERIENCE OF USERS 



Experience Level 





Low 
(N = 2) 


Medi um 
(N = 7) 


High 
(N=7) 


Average Rating, 
Simulation Factors 


13.1 


10.7 


14.0 


Average Rating, 

Training and 

Si tuati onal Factors 


13.8 


12.2 


13.6 
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DEVICE 15E16: ELECTRONIC WARFARE TRAINER. The 15E16 is used 
to teach all phases of electronic countermeasures to indi- 
vidual operators and electronic warfare officers. It is 
intended to train the principles of operation of various EW 
equipment; skill in the interpretation of displays and read- 
outs; skill in classifying intercepted electromagnetic 
radiations and ability to analyze those radiations in detail; 
recognition of the electronic characteristics of various 
types of radar equipment; and the ability to determine the 
priority of a threat signal. It was expected that the 
trainer would be used by operators with relatively little 
experience as well as provide refresher training to highly 
experienced personnel. 

The device provides a means for synthetically generating 
and distributing a number of signals to each of several user 
ECM stations. The signals are supposed to simulate either 
friendly or hostile emitters, and are to appear not as 
''optimum signals," but as signals that would be received 
under actual operating conditions. 

The qualifications considered necessary for instructors 
using this device are that they be commissioned officers or 
senior petty officers with a "thorough knowledge" of elec- 
tronic warfare, of the capabilities and limitations of 
Device 15E16, and of the corresponding electrpnic counter- 
measures equipment and procedures it is designed to simulate. 

The 15E16 is a trainer that evidently had some early 
acceptance problems but is now highly accepted at Moffett 
Field. Its composite criterion score was +3.5. Figures 6 
and 7 show the acceptance profiles for this trainer. With 
respect to simulation, the 15E16 was rated distinctly above 
the average of all trainers in five areas: external en- 
vironmental effects, equipment performance , control effects, 
display of status information, and simulated sensor reactions. 
It received only one below average rating, that in the area 
of communication problems. However, it is important to note 
the high importance attributed to fidelity of simulation, 
particularly in equipment performance , inf ormat ion disp lays , 
and sensor performance. Thus the profile shows several 
negative "importance differentials," a fact that very likely 
somewhat reduces its level of acceptance. 

With respect to training effectiveness factors, it was 
rated above the average of all trainers as a group on all 
but three factors . In fact, it received only one below 
average rating, on immediacy of performance evaluation. it 
was also rated low on completeness of performance evaluation 
although it did not compare unfavorably with other trainers 
in this respect (Figure 7). 
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The comparatively low rating 
ness and immediacy of performance 
from the fact that the instructor 
well the student is doing without 
each individual student position. 



of the trainer on complete- 
evaluation appears to stem 
is unable to determine how 
getting up and going to 



> 



The qualifications of instructors are considered critical 
in this trainer and it was indicated that training sometimes 
suffers from the unavailability of qualified personnel. 

Although the trainer received an above average rating on 
reliability and maintainability, it is evident that it is not 
without its difficulties insofar as maintenance personnel are 
concerned. The trainer is "pref lighted'' every morning so 
that any problems that might arise are identified before the 
students arrive. However, troubleshooting is complicated by 
incomplete or incorrect documentation; the documentation for 
the trainer as a whole is the subject of considerable mis- 
trust. Further, the modified GFE is not covered in all the 
documentation- The master scan generator is considered a 
source of particular difficulty. 

It was indicated that some negative attitudes developed 
early toward this trainer because of an unusually loug period 
of time before it was accepted. 

"The allowance parts list did not become avai].?:.ble 
until two years after delivery. In the interim 
several modifications were made which limited the 
value of this list. Many things were missing and 
many added . " 

"Explanations for various systems in the technical 
manuals, drawings and schematics are often too 
brief considering the complexity of the device and 
the lack of any formal maintenance training course. 
There are too many errors in the maintenance 
instruction . " 



Use 
FAETUPAC 
it. The 



of the device is being promoted actively by the 
and by the instructors who are enthusiastic about 
following comments were offered by students: 



"It 
it . 
ing 



is a very 
It's the 



good trainer . 
closest thing 



I learned a lot from 
to being actually fly- 



m 



sure it • s nothing 



like 
time 



that I have seen, 
flying, though. I do have a few hours flight 
in the radar seat, and that was nothing like the 
trainer . " 



52 



1 



|er|c 



61 



NAVTRADEVCEN 70-C-0276-1 



"I had heard that this was a million dollar trainer, 
that * it was quite elaborate and supposed to be one 
of the best around. I was very impressed when I 
first walked into it. It made me get the feeling 
that I couldn^t wait to use it." 

The need for training in the recognition of individual 
signal sources is clearly recognized: 

"You can never get enough experience in the iden- 
tification of possible enemy emitters. More time 
should be spent listening and watching emitter 
s i gnals . " 

A shortcoming of the device was considered to be the 
absence of cameras for taking pictures of signals and re- 
corders for tape recording the signals: 

"In the actual aircraft you must use thes e 
(recorders) . " 

A point repeatedly emphasized was the inability of the 
instructor to monitor the performance of six students simul- 
taneously and to receive all of the reports that they might 
initiate for evaluation. It was felt that communication 
between the operators and the instructor was much less im- 
mediate than in the case of actual operations where communica- 
tions between operator and TACCO is nearly instantaneous. 
These criticisms probably are reflected in the somewhat lew 
average rating given the trainer in communication procedures. 

The reputation of the trainer insofar as signal realism 
is concerned was mixed. While some held that the trainer 
was adequate for training in the recognition of individual 
signal emitters, there were others who felt "it is good for 
learning operation of the gear, but a waste of time as far 
as individual signal recognition goes." One problem men- 
tioned was the absence of simulated background noise asso- 
ciated with the aircraft. Generators and other equipment 
radiating electronic noise reportedly complicates the prob- 
lem of ECM operation in the aircraft in a way that is not 
reflected in the trainer. 

The number of signals made available by the trainer at 
any one time was considered to be a limiting factor. In 
contrasting the 15E16 with the 15E18, it was pointed out that 
in the latter trainer the computer has "every" signal avail- 
able and can compound the problems that the operators will 
be expected to cope with during an actual mission. 
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It was also felt that there was insufficient accuracy 
in target course, speed and position information. This 
added to the difficulty of evaluating student performance. 
In addition, lack of provision for tape recording target 
information led to an inability to demonstrate the importance 
of tapes to the student and a limitation on evaluating his 
per f ormance . 

Several comments were made concerning the lack of cor- 
respondence between the arrangement of equipment in the 
trainer and that in the aircraft: 

''Designers made gear installation as convenient 
rather than simulating real installations. It 
doesn't prevent good instruction, just realism." 

It was felt that an improvement would be to arrange the equip- 
ment in a manner that it would be identical to that in the 
aircraft. It was also suggested that all associated controls 
should be represented even though they might have no function 
in the trainer. These included such controls as the tape 
recorder control box, ASA-16 marker panel, ECM audio switch, 
e t c . 

In general, it was felt that the trainer was fulfilling 
a most important role: 

"It gives the operator a simulated contact and can 
show his capabilities and limitations. Without 
the trainer the operator might detect a contact in 
the real world and never know what it is. It 
provides training in a field the operator could 
not get otherwise . " 

It was clear that the instructors are playing a very 
positive role in publicly promoting the value of this train- 
ing device and took considerable pride in it being the "best 
in the training environment" at Moffett Field. It was gen- 
erally felt that use of the trainer should be increased 
through, for example, the use of "mandatory operator profi- 
ciency courses" but that there were insufficient instructors 
at this time to permit such usage. 

One instructor who had over 1200 hours of operator ex- 
perience in the P3B, most at the ECM station, strongly 
emphasized the importance of electronic background noise 
experienced in the aircraft itself. The noise emanates from 
the aircraft generators and other equipment and there is 
also a great deal of weather noise in the form of static 
electricity. It was felt that a provision should be made to 
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simulate this problem in the 15E16, in the form of a variable 
noise control, so that students ''would knew what the real 
world situation would be like and get used to analyzing sig- 
nals under adverse conditions . " 

It was evident that the 15E16 at one time had a very 
poor reputation with respect to reliability. However the 
trainer at Moffett Field has been kept in an excellent state 
of repair by the maintenance technicians. Nonetheless, there 
were numerous complaints about the difficulty of conducting 
maintenance, performing calibrations and alignments, and the 
modi fi cations necessary in the trainer to accept government 
furnished equipment • Schematics and interconnect diagrams 
were reportedly extremely difficult to follow from system to 
system and there is an absence of overall system block or 
flow diagrams. Explanations in the maintenance textbooks 
were considered too brief in relation to the complexity of 
the device and the absence of any formal maintenance training 
course • Maintenan ce instructions were reportedly replete 
with errors. Thus, though the equipment was generally regarded 
as reliable, it was also regarded as very difficult to trouble- 
shoot • 

Evidently, the trainer's reputation with respect to poor 
reliability stemmed from an installation on the Atlantic Coast 
where, following a major modification of the device, main- 
tenance personnel found themselves "insufficiently qualified 
due to a lack of interest and/or participation in the modifi- 
cation," Consequently, trainer reliability proved very poor. 

It is evident that the FAETUPAC Detachment at Moffett 
Field is doing a systematic promotional job with respect to 
this trainer. It was reported that they have held open 
house and invited squadron commanding officers and electronic 
warfare officers to a presentation on the capabilities and 
limitations of the device. This has led to a renewed interest 
at the squadron level and higher morale among the instructors 
who teach ECM. It has also helped overcome an initial nega- 
tive viewpoint associated with the eleven-month period of time 
prior to acceptance of the trainer. During this time NTDC 
reportedly instructed the technicians not to get involved with 
the equipment because it was felt the contractor might use 
this as an excuse for subsequent system failure. 

Table 7 shows that there were no systematic differences 
in acceptance of the 15E16 associated with amount of opera- 
tional experience. However, experienced personnel rated it 
much lower than inexperienced personnel with respect to the 
level of training for which it is suited. 
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TABLE 7. AVERAGE RATINGS OF 15E16 BY LEVEL 
OF OPERATIONAL EXPERIENCE OF USERS 



1 



Experience Level 





Low 
(N=8) 


Medi um 
(N = 3) 


High 
(N = 7) 


Average Rating, 
Simulation Factors 


10,4 


12.7 


11.2 


Average Rating, 
Training and 
Situational Factors 


10,4 


11.5 


10.7 



DEVICE 14A2: ASW TEAM TRAINER, The 14A2 is a heavily uti- 
lized team trainer for surface ship ASW operations* It has 
simulated spaces for the bridge, CIC, sonar, and weapons 
control aboard typical destroyers. It is equipped with sim- 
ulated AN/SQS-23 sonar, MK 111 and 114 fire-control systems, 
fire-control radar, weapon control and launcher control sta- 
tions, and a target control station manned by the trainer 
operator. The entire complex is driven by a general purpose 
digital computer. 

The 14A2 is one of the most highly accepted trainers in 
the Navy, It received a composite criterion score of 6,4, 
the highest of any trainer in the study. 

The simulation profile for the 14A2 is shown in Fig- 
ure 8, It is evident that the trainer is considered superior 
with respect to simulated performance of the equipment and 
vehicle, the effects of controls, communication procedures, 
and weapon control. It was considered about average, al- 
though somewhat below judged importance, in display of status 
information, sensor performance, and target reactions. The 
size of these discrepancies was comparatively small, however, 
so that over al 1 the trainer received positive standard scores 
on simulation factors (see Table 4), 

The 14A2 was rated consistent ly above average on train- 
ing effectiveness factors. This was particularly true in 
the areas of software and performance evaluation (see Fig- 
ure 9), It is believed that these superior ratings reflect 
an unusual investment in the development of utilization 
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guides and methods of performance appraisal associated with 
this trainer,* In fact , in interviews with personnel op- 
erating the 14A2F (Pearl Harbor), the officer in charge 
stated that "we would have been lost without it (the uti- 
lization guide) at the beginning," 

Some observations with respect to the simulation pro- 
file appear in order. Performance simulation was rated very 
high , in part , perhaps because the 14A2 equipment actually 
operates "better than operational equipment," It is more 
accurate , more effective (fewer missed targets --better 
range--no detection uncertainty), and suffers no equipment 
casualties, (If a casualty occurs, the exercise is inter- 
rupted until it is fixed.) 

Although the majority of respondents rated control sim- 
ulation very highly, it is interesting that the sonar 
technicians, as a group , disagreed with the overall judgment. 
That is, they did not think simulation was good enough. 
This reflects their appreciation of the problems of operating 
techniques in the real world. Several operators commented on 
the lack of provision for developing contact classification 
information. This criticism is also reflected in the com- 
paratively low rating awarded to sensor simulation for this 
trainer. It is ironic, perhaps, that some of the acceptance 
of the 14A2 certainly stems from the fact that sonar simula- 
tion was much more sophisticated in this trainer than in 
earlier attack teachers , Nevertheless, the fleet has come 
to recognize the importance of high fidelity sonar simulation, 
i,e,, classification quality. 

In addition to the "low quality" of sonar simulation, 
it is perhaps useful to point out that no attempt was made 
to provide realistic radar simulation. The complaint reg- 
istered most often in relation to radar simulation was that 
it made ASW air control too easy. 

The ratings with respect to target reactions probably 
would have been higher except that some instructors still 
"play" with the student since target movement is directly 
under his control, A related shortcoming was felt to be 
lack of realistic countermeasure devices, beacons, false tar- 
get cans, decoys, etc. 

The high rating on weapons firing and control also 
deserves comment, A maj or . ob j ective in the development of 



*See Device 14A2 (Series) Utilization Guide for Surface Ship 
ASW Attack Team Training, Dunlap and Associates, May 1969, 
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the 14A2 was to provide realistic training in the firing of 
ASROC weapons. The rating on this item reflects the achieve- 
ment of that goal. Some dissatisf acti on was expressed with 
the fact that the trainer has no provision for torpedo kill 
probability, even though "not all well-placed torpedos result 
in hits and not all torpedos run hot, straight, and normal." 

The high ratings on completeness and immediacy of perfor- 
mance evaluation probably reflect the somewhat special situa- 
tion where a complete evaluation team of six or eight people 
is available. It is felt that the ratings would have been 
much lower if they reflected the more typical situation where 
only a single instructor, seated at a remote station, is 
prepared to offer post -exercise evaluative commentary . 

It is believed that the comparatively low rating on value 
of time in the trainer probably reflects a number of considera- 
tions. The lack of high fidelity sonar simulation has already 
been mentioned and probably contributed to this rating. In 
addition, any simulated ASW exercise of necessity involves a 
certain amount of time during which nothing of great signifi- 
cance seems to be happening. Although this is a necessary 
element in realistic search plans and contact developments, it 
is possible that this is construed as time wasted by some 
personnel. Finally, and perhaps most important, some personnel 
reportedly felt that the exercises in the trainer do not ad- 
vance to a very high level of complexity. Interestingly, the 
instructors sometimes claim the students are not ready; the 
students say the instructors cannot, or do not want to, develop 
highly complex problems. 

Despite these identifiable problems, in general it is con- 
cluded that the 14 A2 represents one of the most highly accepted 
and certainly most highly utilized training devices in the Navy. 

There were no systematic differences in level of acceptance 
as a function of the user*s level of operational experience. 



TABLE 8. AVERAGE RATINGS OF 14A2 BY LEVEL 
OF OPERATIONAL EXPERIENCE OF USERS 

Experience Level 





Low 
(N = 5) 


Medi um 
(N=13) 


High 
(N=12) 


Average Rating, 
Simulation Factors 


1 1 .6 


10.6 


10.6 


Average Rating, 

Training and 

Si tuati onal Factors 


1 1 . 1 


1 1 .0 


1 1 .6 
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DEVICE 21A38: SUBMARINE ATTACK TRAINER, The 21A38 is a 
shore-based attack trainer comprised of three independent 
attack centers that share a common computer facility for 
problem generation and evaluation. The trainer is designed 
for the indoctrination and training of submarine fire con- 
trol teams in (1) the basic characteristics and capability 
of fire-control techniques and equipments utilized in sub- 
marine ASW; (2) basic techniques involved in the approach 
and attack of single and escorted surface targets; (3) ad- 
vanced tactical training in both pack and submarine versus 
submarine techniques ; (4) the setup and firing of submarine 
weapons; (5) the integration of sonar equipment and operating 
personnel into the fire-control team; and (6) the possible 
development of new tactical doctrine. 

The acceptance profile of the 21A38 is complicated by 
the fact that it is essentially three trainers in one* 
Attack Center No, 1 is designed to simulate modern diesel 
submarines and early nuclear submarines. It is equipped 
with corresponding sensor and. fire-control equipment (BQS-4, 
BQR-2B, BQG-4, and MK 101 Fire-Control System), 

Attack Center No, 2 is designed to simulate the fast 
attack submarine. It is equipped with the relatively more 
modern BQS-6B, BQR-7B, and BQH-2C sonar systems. 

Attack Center No, 3 is designed to simulate the ballistic 
missile submarine. It is equipped with the BQS-4, BQR-7B, and 
BQR-2B sonars and the MK 113 Mod 7 fire-control system, 

A difference of fundamental importance between the three 
attack centers is the presence of a periscope simulator which 
is a feature only of Attack Center No, 2, This has been a 
major factor in acceptance as reflected by steadily increasing 
utilization of Attack Center No, 2 since the periscope instal- 
lation, and corresponding decreases in utilization of the 
other attack centers. Because it was reported to us that the 
periscope simulator was a major consideration in acceptance, 
and because of the relatively few available users of Attack 
Centers 1 and 3, the majority of the acceptance data gathered 
during this study reflects user appraisals of Attack Center 
No, 2, This portion of the device is highly accepted with a 
composite criterion score of +5,0, 

The simulation profile for Device 21A38 is shown in Fig- 
ure 10, The device is rated as superior in its simulation of 
the internal operating environment , performance of the equip- 
ment and vehicle, effects of controls , and weapon firing and 
control. Felt simulation deficiencies are reflected in the 
display of status information, sensor performance, and target 
reactions , 
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In addition to the periscope requirement, the most fre- 
quently mentioned deficiency of the 21A38 has been the lack 
of realistic sonar simulation. Numerous complaints were 
registered by fleet personnel concerning the quality and 
variety of target sounds. Despite the critical role of the 
submarine sonar operator with respect to target detection, 
classification, and motion analysis, personnel with train- 
ing responsibility aboard some submarines consider it a 
waste of time to send their sonar operators along with the 
attack team to the 21A38. In addition to poor audio simula- 
tion, the absence of a LOFAR type display is a source of 
complaints. Few of the operational problems encountered at 
sea are well simulated. The result is that the sonar opera- 
tors feel they play only a minor role in the trainer and, 
once in the "automatic target following" mode, are called 
upon to do little but sit in their chairs. 

The comparatively low rating on sensor simulation also 
reflects dissatisfaction with the periscope. The quality 
of target imagery was felt by some to be not as good as that 
in some earlier attack trainers. This makes "target iden- 
tification more difficult that it is in the real world." 

The training effectiveness profile of the 21A38 is shown 
in Figure 11. It is clear that the trainer was rated par- 
ticularly high in problem setup, reliability, and level of 
training; it was also above average in most other respects. 

The only below average ratings of the 21A38 occurred in 
the areas of performance evaluation and value of time in 
the trainer. There is an administrative problem that is 
probably responsible for these reactions. There is no stan- 
dard syllabus or graded problem series through which submarine 
attack teams are regularly exercised. Rather, the facility 
tries to respond with the particular types of problems that 
a given team may request. While these requests almost always 
can be accommodated in a general way, the burden of evalua- 
tion and critique often falls on enlisted instructor person- 
nel who are not fully qualified to assume these roles. This 
stems in part from their lack of complete technical knowledge 
about all stations in the trainer, in part from their lack of 
detailed information about various types of targets, and in 
part from the social difficulties of critizing one of higher 
rank than oneself. 

It is interesting zo note that although the trainer has 
a capability for submarine versus submarine action, with 
different submarine teams operating in the different attack 
centers, the trainer has rarely been used in this mode. In 
one exceptional case, two submarine teams were pitted against 
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one another without the knowledge of the commanding officers. 
The negative reaction of the commander whose submarine lost 
the contest has evidently created a lasting attitude of 
rejection insofar as this particular feature cf the trainer 
is concerned. 

There is a general feeling at the training center that 
qualified observers at the squadron level should participate 
more extensively in performance evaluation in the trainer. 
It was asserted that use of the trainer is diagnostic in that, 
after one or two runs, a qualified observer can readily deter- 
mine what the deficiencies of that crew are. Further, pre- 
dictions can be made concerning what types of difficulties 
the crew will have, given a particular tactical situation. 

In summary , despite some notable deficiencies in the 
minds of the users. Device 21A38 must be regarded as a 
highly accepted trainer. Personnel at the training facility 
are actively supporting its use by the fleet and fleet accep- 
tance appears to be generally high. To the extent that there 
are negative reactions, they are associated most strongly 
with inadequate sonar and periscope simulation and with 
deficiencies in performance evaluation and supporting soft- 
ware. There were no significant trends in acceptance as a 
function of experience level (see Table 9) . 

TABLE 9. AVERAGE RATINGS OF THE 21A38 BY LEVEL 
OF OPERATIONAL EXPERIENCE OF USERS 



Experience Level 





Low 
(N = 6) 


Medl urn 
(N=4) 


High 
(N=15) 


Average Rating, 
Simulation Factors 


10.3 


10.2 


10.7 


Average Rating, 
Training and 
Situational Factors 


11.4 


9 . 6 


10.8 



TRAINERS WITH ACCEPTANCE PROBLEMS 



It has been emphasized that probably no trainer repre- 
sents a case of complete acceptance or rejection. Each 
trainer, whether generally accepted or not, may have its strong 
and weak points in the minds of the users. However, a number 
of trainers did show evidence of lack of acceptance with re- 
spect to many if not the majority of the characteristics 
studied . To emphasize the contrast between "accepted" and 
"non-accepted" trainers, a number of these will be described 
here . 
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DEVICE 2F55B: F4B CREW TRAINER. The 2F55B was designed to 
provide training in aircraft control , instrument procedures, 
engine control, emergency procedures, ECM operations, and 
coordination between pilot and RIO (Radar Intercept Officer) 
in the F4B aircraft. 

The F4B is a supersonic, two-place, twin-engine, all- 
weather fighter aircraft. It was designed for intermediate 
and long-range high-altitude intercept work, using the 
APQ-72 and APA-157 radar sets, and for intermediate or long- 
range attack missions for conventional or special weapons 
delivery. The responsibilities of the RIO are to manage the 
airborne missile system, communications, navigation, and ECM. 

The components of the navigation and bombing systems, 
which are viewed, controlled or operated by flight personnel, 
were to be "identical in appearances and operation" with the 
actual equipment. The aircraft is designed to carry a vari- 
ety of missiles whose simulation was to accurately reflect 
flight characteristics, weight, and center of gravity changes. 
Simulation of launching, missile firing, and kill conditions 
was to be provided. Target performance was to be simulated 
on the radar indicators in a "realistic" fashion. In addi- 
tion, "maximum implementation was to be made of human factor 
considerations in the design of the trainee and instructor 
stations to insure optimal training value." 

Among the trainers designated as low acceptance cases, 
the 2F55B was the most marginal. Its composite standard 
score, -2.7, was not greatly below average, although it 
received negative scores on most of the elements making up 
the composite (see Table 4). 

The simulation profile for the 2F55B (Figure 12) iden- 
tifies the felt deficiencies in greater detail. It is clear 
that the trainer was rated below average on about half of 
the simulation factors but particularly on performance of the 
equipment, effects of controls, simulation of target reac- 
tions, and weapon firing and control. In addition, the 
trainer was judged to be significantly deficient, compared 
to judged importance, in its simulation of status displays 
and sensor operations. 

With respect to training effectiveness factors, the 
profile of the 2F55B (Figure 13) shows felt deficiencies in 
the area of software, level of training, comprehensiveness 
of training, the value of repeated use, and in overall com- 
parison with similar trainers of this type. To understand 
this profile, the following observations from the interviews 
are offered. 
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It is "a poor package of hardware and there is poor 
documentation on the software. The trainer will not perforin 
as a weapons system trainer. The radar and OFT portions can- 
not be operated as a package. The OFT definitely does not 
fly like the F-4 aircraft as far as general feel is concerned. 
The first trainer arrived shortly after the aircraft but was 
not operational for about eight months. The first trainer 
was designed around wind tunnel data and the aircraft proved 
to be quite different. The trainer then had to be returned 
for rework. This trainer was particularly bad and the whole 
program suffered because of it. Attitudes are just now 
changing because maintenance personnel have really worked to 
get it back in top shape." 

Although it was felt that the flight equations eventually 
designed into the trainer were accurate and complete, it 
reportedly has been difficult to maintain the proper adjust- 
ments because of maintenance and the unavailability of parts. 
Because of insufficient air-conditioning, the trainer has had 
to be operated under adverse conditions. The packaging and 
hardware are considered to be of inferior quality and a major 
cause of troub leshooting prob lems . Drawings and s chemati cs 
are reportedly incomplete ; no flow diagrams were provided 
with the trainer with the result that maintenance people had 
to spend many hours developing them. Signal paths were re- 
portedly difficult to follow. 

While a major drawback to the trainer is associated with 
its handling characteristics and flying performance, it was 
also stated that the tactics portion does not meet local 
requirements . 

"The present trainer cannot track on multiple 
navigational aids, TACAN, etc., without great dif- 
ficulty on the part of the operator. Although 
some RIO students feel that the trainer is good 
for knobology, ECM review, and navigation review, 
they tend to feel that there is insuf f i ci ent reward 
for the time spent in the trainer. They feel that 
they often just sit there while the pilot plays 
with the trainer. They also complain that the 
radar cockpit systems are not compatible with the 
current cockpit confi gu rat ion . " 

Coupled with the complaints about the hardware, there 
were very serious reservations about the quality of instruc- 
tion associated with this trainer. It was asserted that the 
instructors do not take the time to find out what the device 
can and cannot do. Some of the instructors were not con- 
sidered up-to-date on emergency procedures. Many students 
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felt that instructor pilots should operate the trainer rather 
than training devicemen. It was complained that unreal or 
unrelated emergencies were given to the pilot simultaneously 
and that this generates a bad attitude toward use of the 
trainer. Further, it was felt that the instructors are often 
not thoroughly familiar with what occurs in the air and how 
the aircraft responds to these occurrences. The complaint 
was summarized in the assertion that "the things taught in 
the trainer are too simple." 

On the positive side it was stated that, "combined with 
a working radar and fire-control system, this F-4 trainer 
could be utilized extensively to great advantage. The 
trainer is worthwhile but by no means is it used to do all 
of the things it was designed for. It works adequately in 
many ways but the RIO, in particular, is being shortchanged." 

It was evident that little promotional effort was being 
made to "sell" the trainer to its potential users. The 
instructors expressed a need for more access to the squadrons 
so that they could determine what was desired in the way of 
training. They indicated that, in the absence of any instruc- 
tions requiring use of the device as a complete weapons 
systems trainer, a trend has set in to use it as an emergency 
procedures trainer only. 

No standard of qualifications is set for instructor 
personnel with the result that there are some pilot person- 
nel who are considered quite acceptable as instructors but 
other non-pilots who are not. It was asserted that "senior" 
instructors have an unfavorable opinion of the trainer and 
that this opinion has spread through the chain of command 
and to their flying peers. There was, however, no identifi- 
able difference in overall acceptance as a function of level 
of experience with the F4B (see Table 10) . But highly ex- 
perienced personnel did rate the trainer considerably lower 
with respect to both value of time spent in the trainer and 
value of repeated use. The same personnel rated it higher 
than neophyte personnel in terms of completeness and im- 
mediacy of performance evaluation. Perhaps this was because 
they performed their own evaluations. 
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TABLE 10. AVERAGE RATINGS OF THE 2F55B 
BY LEVEL OF EXPERIENCE OF USERS 



Experience Level 





Low 
(N=9) 


Med i urn 
(N=10) 


High 
(N=3) 


Average Rating, 
Simulation Factors 


9.0 


9 . 0 


8.9 


Average Rating, 
Training and 
Situational Factors 


8. 1 


10.0 


8.9 



DEVICE 2F69A: P-3 AIRCREW TRAINER. The 2F69A was designed 
to simulate the P-3A antisubmarine patrol aircraft, its 
associated sensors and weapons sytems • It was expected to 
(1) simulate the actual aircraft in both normal and emer- 
gency flight conditions; (2) permit training in the use of 
all radio navigational equipment for takeoff and en route 
flying, approach, and landing problems associated with 
instrument flying; (3) permit training in the use of all 
tactical equipment installed in the actual aircraft; and (4) 
teach team coordination, team training, and ASW tactics for 
the entire crew of the P-3A. 

It was assumed that the pilots, engineers, and operators 
using the trainer would be previously checked out in the 
basic operating procedures for their respective equipments • 
For this reason, the military specification indicated "that 
the detail and realism often associated with operator 
trainers is not required. However, the device must provide 
all the inputs with sufficient realism so that the crew 
members can make the necessary tactical decisions." 

The level of training associated with this device was 
described as "advanced." The device was expected to "knit 
the crew into a team which will permit them to operate the 
aircraft for its mission purpose. Through canned and dynam- 
ic problems they will learn to work together, exchange 
information, and search, detect, locate and attack simulated 
targets. In this trainer they should reach a level of pro- 
ficiency enabling them to conduct actual ASW operations at 
sea in their aircraft." 
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The 2F69A has fallen well short of these objectives in 
the view of many users- Its score on the composite accep- 
tance criterion was -3.5 (see Table 4). It is evident from 
the profile of simulation factors (Figure 14) that there 
were serious differences between the trainer as designed and 
the felt importance of various simulation factors. Deficien- 
cies are particularly evident in the ratings of vehicle 
performance, effects of controls, display of information, 
sensor performance, target reactions, and weapon firing and 
control . 

Notable deficiencies are also evident in the profile of 
training effectiveness (Figure 15) . Particularly low ratings 
were given in problem setup, software, and reliability. Below 
average scores were also given on provisions for performance 
evaluation. These deficiencies did not preclude, however, 
relatively high ratings with respect to the value of trainer 
use (in comparison to opportunities in the operatinal air- 
craft) and the value of repeated use of the trainer . In the 
commentary that follows, an attempt is made to convey some 
of the reasons given for these quite different viewpoints. 

"The tactics portion of the trainer has never been 
used since it arrived at Barber^s Point. Prior to 
that time it had operated in a coup led mode for 
about two years at Moffett. It was good in concept 
but the tactics portion was highly unreliable. It 
was down in some respect during every other problem, 
requiring as much as two maintenance shifts to keep 
it operating. " 

"The JEZ simulation was always very poor, never 
really accepted. It took seven months for them 
to go through a three months acceptance program. 
NTDC acceptance checks were largely static; many 
of the tests met the specs but didn't really 
tell you how it would operate dynamically. The 
result is that the 14B31 is used for tactical 
training instead of th^ 2F69A. This is a very 
simple trainer utilizing manual inputs to the 
JULIE, JEZEBEL, and TACCO operators." 

Some squadron personnel contended that they got more 
training done with the two sections of the trainer (OFT and 
Tactics) separated. 

"There is no need to get the whole crew to the 
trainer . We would , however , like to use the 
2F69A as a full weapons system trainer for one 
reason--to demonstrate the qualifications of a 
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particular crew member as a member of a coor- 
dinated crew. We can do this much more econom- 
ically in a WST. We have 23 qualification 
exercises that must be achieved by each crew. 
We frequently can run the necessary exercises 
with only a partial crew in the trainer." 

The software associated with the trainer was the subject 
of considerable criticism. The training guides were described 
as very sketchy. 

"We couldn't train simply using the guide. We 
(the training devicemen) learned to operate the 
tactics portion by observing the squadron in- 
structor. The operator's guide we have is at 
the bottom of a drawer. The instructors gen- 
erally get acquainted with the trainer's 
capabilities by asking the operator 'Can you 
do this?'" 

Personnel in charge of the trainer had developed descrip- 
tions of all of the classes of problems they could generate, 
but they still depended on squadron personnel to define the 
manner in which they wished to use the trainer. The opinion 
was strong that they needed a qualified instructor attached 
to the training group. The fact that the training devicemen 
in the training group were not "NATOPS qualified" was a 
matter of some concern. It was felt that a syllabus should 
be developed by qualified people and then "...laid on the 
various squadron training officers. At the moment, we don't 
know what these people need." 

It was evident that the trainer was being used to train 
flight engineers in emergency procedures more than for any 
other purpose. This was one purpose for which the trainer 
was felt to be extremely valuable. An attempt was made to 
schedule trainee engineers for the same exercise they would 
have the following day in the aircraft. It was felt that 
they could set up many problems that cannot be practiced in 
the aircraft with safety. However , the f 1 i ght engineers 
complained that there were many system malfunctions and 
checks that they could not set up. They considered the 
trainer generally deficient in the area of simulating fail- 
ures and malfunctions. It was stated that they frequently 
have to "...jerry-rig or cut out something in order to get 
the desired effects. It should be possible just to push a 
button to get the desired setup." A related problem was 
the inability to simulate sequential effects: "If one prob- 
lem develops in the real world, another related problem is 
very likely to appear also. There is insufficient provisi on 
for this type of thing in the trainer." 
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It was evident that the trainer was being used a sub- 
stantial proportion of the time in the absence of not only 
the tactical portion of the crew, but the pilots as well. 
One squadron engineer stated that they only have a pilot in 
the trainer about one-third of the time that the engineers 
are there. Therefore, "If the autopilot portion of the 
trainer is not working (and it frequently is not) , then we 
cannot train. Sometimes the flight engineers end up flying 
the aircraft themselves." 

In summary, it appears that the 2F69A has suffered ex- 
tensively from an inability to keep the tactical portion of 
the trainer operational and, as a direct consequence, has 
been rejected in its designed role--that of a vehicle for 
coordinated training of the entire crew. In addition, there 
is concern over the ability of the instructor personne 1 to 
achieve the full potential of the trainer. 

On the positive side, it was stated that before the 
trainer became available, transition from another type of 
aircraft to the P-3A was particularly difficult. It was 
necessary to learn all the required procedures and sequenc- 
ing in actual flight--"a very tough proposition." 

One training officer claimed that the users simply do 
not appreciate what they have in the trainer- -they expect 
too much. The same individual asserted, however, that the 
2F69A was a procedures trainer and that the pilots cannot 
expect it to fly like the real thing. Such a view is hardly 
in accord with the objectives set forth in the military char- 
acteristics for the P-3A trainer series. Certainly a device 
limited to procedures training, whether it be for pilots, 
flight engineers, or tactical personnel, could be achieved 
with far less complexity and investment than the 2F69A. As 
a procedures trainer, it may be accepted; as a complete 
weapon systems trainer, it obviously is not. 

Unlike the other trainers discussed thus far, the 
ratings of the 2F69A suggested a general relationship be- 
tween trainer acceptance and level of operational experience. 
Generally, the more experienced the participant was in P-3A 
operations , the more critical he was of the trainer (see 
Table 11). Criticism of the most experienced personnel was 
directed primarily toward vehicle performance, displayed 
information, sensor performance, problem setup, software, 
reliability, and overall comparison with similar trainers. 
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TABLE 11. AVERAGE RATINGS OF THE 2F69A 
BY LEVEL OF EXPERIENCE OF USERS 



Experience Level 



< 


Low 
(N=4) 


Medi um 
(N=7) 


High 
(N=7) 


Average Rating, 
Simulation Factors 


10.6 


10.5 


8.6 


Average Rating, 
Training and 
Situational Factors 


10.2 


9.7 


8.8 



DEVICE 2F69B.-S0ME CONTRASTS WITH THE 2F69A. Very likely in 
recognition of many of the problems cited in connection with 
the tactics portion of the 2F69A, the specification for 
Device 2F69B required that the trainer provide "realistic and 
accurate simulation for the search, detection, localization, 
and attack functions of the P-3B antisubmarine warfare air- 
craft." There was to be simulated activation of all controls, 
instruments, gauges, recorders, plotters, indicators, control 
panels, communication systems, lighting systems, and the like 
associated with MAD, JULIE, JEZ, ECM, navigation, and the 
tactical coordinator station. All modes of operation were to 
be simulated. 

Provision was to be made at the instructor's control 
station for the development of targets having carefully 
specified underwater sound characteristics and signatures, 
emission characteristics, mode of operation , performance , 
and other target type characteristics. Detailed specifica- 
tions were also provided for ocean environmental effects 
that affect signal propagation and display. 

A comparison of the acceptance criteria for the 2F69B 
and 2F69A suggests that a number of significant improvements 
were, in fact, achieved. The composite criterion score for 
the 2F69B was +2.2 compared with -3.5 for the 2F69A. In- 
spection of Table 4 reveals that the 2F69B received much 
better scores on simulation deficiencies than the 2F69A. 
Unlike the 2F69A, it is being used as a complete weapons sys- 
tem trainer and enjoying considerably higher utilization. A 
comparison of acceptance profiles shows that the 2F69B was 
rated considerably higher than the 2F69A with respect to the 
display of information, sensor performance , target reacti ons , 
and weapon firing and control (Figure 16), It was also rated 
somewhat higher in equipment/vehicle performance. It is 
noteworthy, however, that in none of these areas was the 2F69B 
rated as high as the felt importance level of these simulation 
factors . 



NAVTRADEVCEN 70-C- fIZye- 1 




78 

87 



NAVTRADEVCEN 70-C-0276-1 



In comparing ratings on training tf f e ct i veit ess factors 
(Figure 17), it is evident that the 2F69B is still felt to 
have serious deficiencies in the areas of software and the 
completeness and immedi acy of performance evaluation. In 
most other respects, it is similar to the 2F69A and closely 
approximates the general average for all trainers. Although 
the 2F69B is much more heavily utilized than the 2F69A, it 
still lacks endorsement in important ways from its users. 
The following comments are offered as illustrative of the 
pr ob lems : 

"Most instructors are familiar with aircraft opera- 
tion but limited in their knowledge of the capa- 
bilities of the training device. The instructors 
are not always well-qualified and current with 
respect to tactics. ASW tactical techniques have 
changed dramatically in the last three years. The 
basic design of the trainer is satisfactory for 
the techniques of five years ago but it is out- 
dated today . If the changes recommended for up- 
dating the trainer (were accomplished) , the atti- 
tudes toward the trainer would change 

"Some instructors appear to lack know ledge of the 
trainer and the exercises that can be provided by 
it. Where instructor operators are also mainte- 
nance personnel, general experience has shown that 
many are not interested in much except the trainer 
upkeep . " 

"Some instructors don't give the student time to 
learn a basic skill before simulating emergencies." 

"The trainer does not fly like the real aircraft. 
It is a good trainer as it is used to teach check- 
list procedures and so forth. Although it does 
not fly like the aircraft, it does not really need 
to do so to accomplish training objectives." 

"The most serious deficiency of the trainer is in 
JEZ training, but there is no other device avail- 
able to do it." 

"Simulation of underwater sound is also a problem. 
Due to poor design, the sound signals drift in 
volume, frequency, and everything else, and must 
be continually readjusted to keep them realistic." 

"The schematics and drawings are insufficient to 
familiarize maintenance personnel with the functions 
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of the trainer. The manuals are generally in- 
adequate in their exp Ian at ions of the functions 
of various systems. A technician has to super- 
impose his own judgment on the adj ustment of 
some of the simulation systems • " 

"Important topics in the manual such as calibra- 
tion procedures and setup procedures, along with 
a detailed theory of operations, are omitted. 
Not all subsystems of the trainer are covered by 
drawings and schematics. For example, special 
weapons, D to A converters, LTV plotter cabinet, 
and subsystems for the land mass generation are 
missing • " 

"It takes a long time for someone to become famil- 
iar with the overall configuration of the trainer 
and how it works, A high degree of specialization 
is required on the part of the individual mainte- 
nance people. When TDs are transferred from one 
trainer type to another, their expertise does not 
necessarily carry over to the new trainer to which 
they are assigned • " 

"Someone is needed in the line of communication 

to report the results of complaints and try to 

see that the proper things are put on the trainer." 

"There have been instances when paper work has 
been submitted for a modification to the trainer 
and there has been no word or feedback at all on 
the submission. A direct line of communication 
with the modification and acceptance group of 
NTDC is needed to find out what has transpired 
for f ol low-up • " 

"The OFT needs a complete change in the aero system 
computations to assure good simulation. A quality 
assurance checking procedure should be set up for 
the trainer, run by the people who fly the aircraft. 
This would assure the quality of the trainer by 
checking systems not in use for everyday training. 
A check crew composed of utilizing squadrons could 
easily be set up and be of benefit to all. They 
could learn of the various trainer capabilities and 
(use) this information (to achieve) a higher degree 
of learning in the squadrons. And the maintenance 
people would know the status of the trainer in 
relation to what the users want and expect." 




NAVTRADEVCEN 70-C-0276-1 



"The trainer provides good experience for basic 
skill acquisition but needs improvement in actual 
instrumentation. Some instruments change too 
fast with power or switch positioning and others 
are too slow. Simulation of the cockpit is good 
and all items are properly located," 

"In general, for the purposes intended, the trainer 
is all right. However, if it was designed to fly, 
it does not do an adequate job in training to do 
this. If the flying characteristics could be im- 
proved, it could help training and flying the air- 
craft for such things as approaches, takeoffs and 
landings , " 

''Performance reports should consist of a detailed 
critique of the entire period and of the actions of 
the individuals involved with the training. Opera- 
tors or maintenance personnel assigned to a trainer 
do not have the means of evaluating a student pilot 
or flight engineer because they lack the specific 
knowledge of flying," 

"The 2F69's reputation with flight crews alternates 
between favorable and unfavorable to a wide degree, 
depending entirely on its use or misuse by the 
period instructor," 

"The trainer is limited only by not having a com- 
prehensive program of use that is mandatory for 
every one , " 

"For the tactics portion, if a kill is made by the 
crew the operators think that it was a good flight 
even if all kinds of errors were made in the tac- 
tical lead -up. They are unqualified to instruct 
each sensor station," 

It was observed that there is a good exchange of infor- 
mation between the FAETUPAC Detachment at Moffett Field and 
the squadron users of this device. It appears that the heavy 
utilization of the 2F69B may be the result of aggressive 
salesmanship on the part of the commander of the FAETUPAC 
Detachment, "He sells the device as a trainer, not as a 
simulator," In the opinion of some, this distinction is an 
important factor with respect to acceptance. It is specu- 
lated that mi litary personnel are more willing to accept 
limitations in a training device than in a unit that is pur- 
ported to be a full-fledged simulator. 
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Table 12 shows that there were essentially no substantial 
differences in acceptance of the 2F69B as a function of opera- 
tional experience in the aircraft. However , the most ex- 
perienced personnel tended to rate the trainer higher in 
communication problems, problem setup, software, and level 
of training. 

TABLE 12. AVERAGE RATINGS OF 2F69B 
BY LEVEL OF EXPERIENCE OF USERS 



Experience Level 





Low 
(N = 5) 


Medi um 
(N=ll) 


High 
(N=14) 


Average Rating, 
Simul ati on Factors 


10.3 


10.8 


10.7 


Average Rating, 
Training and 
Situational Factors 


8. 1 


9.0 


9.5 



DEVICE 14.B35: UNIVERSAL JULIE/JEZEBEL OPERATOR TRAINER. The 
14B35 is a portable, solid-state ASW tactical crew trainer 
capable of injecting dynamic simulated sonar signals into the 
sensor equipment aboard P-3 aircraft. The device is intended 
to provide ASW crews with a means of "increasing their levels 
of proficiency in the detection, classification, and local- 
ization of targets." The 14B35 can be used on board an air- 
craft during flight, on the flight line, in the hangar, or 
as a '*r oof top" trainer for s imul taneous training with several 
aircraft . 

The 14B35 was developed expressly "to improve fleet per- 
formance in the prosecution of JULIE and JEZEBEL contacts, 
to reduce the number of false contacts and false classifica- 
tions." With these objectives, the military characteristics 
clearly reflected the need for realism and sophistication in 
signal simulation "above that currently provided in existing 
part-task training devices." Training was to be directed at 
the intermediate and advance levels and to encompass all 
activities from initial LOFAR detection through final SONO- 
BUOY fixing. 

The instructors for the device were to be "highly quali- 
fied officers and senior petty officers with a comprehensive 
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background in ASW tactics and detection systems." It was 
recommended that the instructor be either a qualified ASW 
pilot or tactical coordinator. All trainees were expected 
to have had basic instruction in the operation of their 
equipment • 

This device, which appears excellent in concept, re- 
ceived next to the lowest composite criterion score of all 
the trainart studied (-7,6), It was rated below average 
on all of tKt elements comprising the composite criterion 
(see Table 4), On the simulation profile (Figure 18), it 
received exceptionally low ratings on display of information, 
sensor performance, and target reactions. It was also rated 
well below the judged importance level on environmental 
simulation (presumably the effects of environment on signal 
present at ion), equipment performance , effects of controls, 
and communication prob lems • 

With respect to training and situational factors, it 
received below average ratings on almost all factors 
(Figure 19), but particularly problem setup, completeness 
of performance evaluation, value of time spent with the 
trainer, value of the trainer versus operational equipment, 
and in overall comparison with similar trainers. 

The lack of adequate submarine target services for ASW 
training is one of the most widely recognized problems con- 
fronting the fleet. It is surprising, then, to find such a 
lack of acceptance of a device that was specifically designed 
.to enable aircrews to use their operational equipment in the 
absence of submarine services. To determine why this was so, 
some of the responses from the questionnaires may be reveal- 
ing. The following comments were offered by instructor 
personnel : 

*'It is a rather complex device in some areas and 
requires sophisticated knowledge in regards to main- 
tenance. The trans mitter section is definitely 
critical 

"The device is limited in that it does not simulate 
DIFAR. It is understood that a modification is 
under development that will permit this capability." 

"The device is not used by deployed squadrons to its 
ful 1 potential . Some instructors are well -qualified 
and some aren't. Some have trouble keeping up with 
the problem. Operators with varying experience are 
employed as instructors--obviously, fleet people 
have better knowledge than others." 
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"The device is reasonably effective for basic 
skill training --however, two of these skills 
will soon be obsolete. Certainly it has no use 
for advance training as it is too dependent on 
human inputs • " 

"A deficiency of the trainer results from the 
necessity of the device operator having to in- 
sert the tactical information manually. The 
communication between the crew and device opera- 
tor detracts from crew performance and problem 



"Due to the manual dependence, I can't think of 
any way of improving this trainer except to 
allow for thirty-one channel capabilities for 
search and one channel for localization. How- 
ever, because the device is used for very basic 
skills, it is not critical that these improvements 
be made. It would probably cost the Navy more 
money than the effort is worth • " 

"The trainer is reasonable for very basic skills. 
The crew is at least in the air and you have 
aircraft movement (in contrast to) the ground 
trainers, I feel it worthwhile in a training en- 
vironment only--I would venture to say there 
would be little use in an operational squadron," 

"Another deficiency is restricted simulation. 
The device provides some unrealistic displays 
but these aren't serious enough to detract from 
the use of the device for basic skills. Remem- 
ber we are talking basic skills and we are 
teaching more equipment manipulation than pure 
tactics , " 

"The device is limited in the number of modes in 
which it can be operated simultaneously. Some 
tactics require that we operate in two modes at 
the s ame time , " 

"The trainer has a poor reputation because every- 
one seems to think it should be just like the 
real thing. They can't seem to accept the fact 
that it is a simulator instead of a duplicator," 

"New operators (acting as i ns true tors) experience 
notable lag periods in making the proper inserts 
to the trainer when working with more experienced 
proficient trainees," 
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"Some skills cannot be trained because oceano- 
graphic phenomena cannot be introduced," 

"Practical JEZ analysis cannot be adequately 
trained because of no ocean hindrances or simu- 
lation of such." 

"The instruct i ons necessary for operat ions are 
often garbled by the inexperienced student (90% 
of our student load), thus hindering proper use 
of the device. The instructions required are 
artificial and distract from good real-world 
oriented training," 

The following comments were offered by students: 

"Equipment malfunctions have limited the device 
in training hours accomplished. Lack of knowledge 
by equipment operators also limits its use, 
Bulkiness and weight do not make it conducive to 
frequent use. On the squadron level it is sched- 
uled very infrequently on the flight schedule, 
(Nine times out of ten this is due to a lack of 
a trained 14B35 operator.)" 

"We never know whether or not it is going to mal- 
function , If it does malfunction, and it does" 
frequently, it always happens in the middle of 
an ASW problem. No one really feels confident 
that the trainer will hold up for the entire prob- 
lem," 

"We have had (trainer) operators who didn't even 
know how to hook up the 14B35 and were actually 
asking the crew members 'what went where?*" 
(However , this s ame respondent felt that it could 
be a good and valuable training aid if used 
correctly,) 

"There seems to be a disparity between what the 
equipment was designed to do and what it has 
actually accomplished. " 

"There are deficiencies (who knows how to fix one 
at the squadron level?) , If the operator doesn't 
know what is going on, what good is the trainer? 
and bulkiness and weight--when you have to lug 
this thing out to the aircraft you think twice 
before you do it . " 
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"The operators do not have enough knowledge of 
tactics to fully operate the trainer t^p the best 
of its capabi lities • " 

"By the time the squadron was proficient with the 
use of the trainer, we had new equipment on board 
the aircraft that would not accept the trainer 
outputs • " 

"The JEZ write-outs are deficient an that there 
are no accurate dynamic changes, very few target 
operating modes , and no aural characteristics," 

"The trainer is very good if used to perfect crew 
coordination. But it does not give the different 
crew members good training in their area of 
operation • " 

"The trainer has a reputation for unreliability 
due to damage that occurs while moving the 
trainer from aircraft to aircraft or back to 
stowage space • " 

"The basic concept of the trainer is right--it 
just doesn*t work all of the time," 

"The trainer operator is required to manipulate 
the trainer so fast on occasions that mistakes 
are made by the operator with low experience 
levels, I think that if some of the controls 
could be operated automatically by the training 
device a more realistic problem could be experi- 
enced by tho trainees. Computerized inputs would 
be highly desirable," 

"The real world has more variables than this trainer 
simulates. Water conditions are quite erratic as 
are submarine s ource levels . " 

"The school for the instructors is too short, I've 
been there. Too much fix it, not enough operate 
it," 

The following comments were offered by maintenance 
pers onnel : 

"Not all computer logic symbols are explained in 
the maintenance manual. Not all equipment needed 
for troubleshooting is available. The parts list 
is incomplete and vague in the description of 
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component parts. Many times parts are not avail- 
able when needed. Maintenance procedures in the 
maintenance handbook are not always correct or 
clear in description," 

"Most of the troublesome problems have been 
rectified. However the transmitter unit requires 
frequent alignment and the PDC switch on the lAl 
console requires frequent rep la cement , " 

"Due to the purpose or function of this device, a 
thorough knowledge of ASW tactics and procedures 
is required (by maintenance personnel). Most TDs 
get no ASW training of any kind in "A" or "B" 
schoo Is . " 

"I would say that it is easy to maintain and a very 
reliable trainer. Also I would say that the East 
Coast FAETUs have had a very poor 14B35 program and 
that the West Coast could give them valuable assis- 
tance in setting up a good workable program," 

"East Coast pe.:sonnel have an unfavorable view of 
the trainer because they were given the 14B35 
devices cold. They received the device (I know 
for I was there at the time) and the manuals and 
were expected to operate it without a school. 
They therefore do not know how to use the device 
with the proficiency that is required to make 
the 14B35 an effective trainer," 

In summary, the 14B35 appears to be a device that was 
excellent in concept but deficient in design for its in- 
tended purpose. The design objectives have been frustrated 
both by basic deficiencies in simulation and by an apparent 
failure by the users to recognize the investment required 
in training personnel for the proper operation of a device 
of this kind. The fallacy that every operator is an expert 
instructor was never more clear. Not only is the device not 
accepted for its intended purpose of advanced training in 
detection and classification, it has perhaps suffered a 
worse fate by becoming thought of as a "basic skill" trainer. 

It is also clear (Table 13) that those personnel with 
the greatest amount of operational experience are the most 
critical of the trainer. They rated it much lower than 
inexperienced personnel with respect to displays, level of 
training, the training opportunities it provided, and value 
of time spent using the trainer. 




NAVTRADEVCEN 70-C-0276-1 



TABLE 13. AVERAGE RATINGS OF THE 14B35 
BY LEVEL OF EXPERIENCE OF USERS 



Experience Level 





Low 
(N=4) 


Medi um 

IN^ 


High 
(N = 5) 


Average Rating, 
Simulation Factors 


8.2 


9.3 


6 . 2 


Average Rating, 
Training and 
Situational Factors 


8.5 


8. 1 


6.7 



DEVICE 2H87: CARRIER LANDING TRAINER. The 2H87 is an air- 
craft carrier approach and landing trainer. The objective 
of its development was to provide a device which could be 
used to train and develop jet pilots in the skills and 
procedures necessary to make safe carrier landings. The 
trainer was to provide training in (1) cockpit procedures 
related to the approach and landing situations; (2) instru- 
ment flight procedures in the carrier vicinity; and (3) 
procedures required for the pilot to successfully land 
aboard a carrier under all visual conditions, day or night 
(from CAVU to minimum visual conditions) • 

It was anticipated that the device would be used in the 
basic training command by students prior to their carrier 
qualifications phase or by qualified carrier pilots to main- 
tain proficiency when shore based. 

The visual presentation was to be such that it would 
"realistically respond to the simulated aircraft movements 
and appear as a carrier would to the pilot whenever it was 
within viewing conditions Geometric distortion and jitter 
in the display were to be minimized to the extent that they 
would not influence trainee acceptability of the device. 

Sufficient resolution was required in the display so 
that the Fresnel simulation would provide usable glide slope 
information to the trainee within one and one-half miles 
from the stern of the carrier under VFR conditions. 

The device was to conform to the T-2B aircraft configu- 
ration except that the full range of T-2B simulated performance 
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in flying qualities was not a simulation requirement. How- 
ever, simulation was to "realistically duplicate the air- 
craft's flight control system in the landing configuration, 
including control displacement, forces, feel and aircraft 
frequency response characteristics," Simulated instrument 
responses were also to duplicate those of the aircraft when 
in this configuration. 

Of all the trainers included in this study, the 2H87 
carrier landing trainer enjoyed the least acceptance. In 
fact, it was not possible to get meaningful evaluations from 
student personnel because, at the time of the study, the 
trainer was being used only as a means of visually present- 
ing the simulated scene during a catapult shot and landing 
as the trainees sat passively in the cockpit. They were 
permitted no active control of the training device at all. 
No trainees who had experienced the trainer used in this 
fashion were available to us at the time of the investiga- 
tion. However it was possible to obtain ratings and inter- 
views with highly qualified administrative, instructor, and 
maintenance personnel. The composite criterion score 
assigned by these personnel was -10,1, The results of their 
ratings are shown in Figures 20 and 21, 

With respect to simulation factors, it is evident that 
the trainer was rated extremely low on virtually every fac- 
tor: internal operating environment, external environment, 
vehicle performance, controls, displays, and sersors. 

The trainer was also rated well below average in most 
training erf ect iveness factors but particularly problem set- 
up, s of tware, comprehens i veness of train.ing, the value of 
the trainer in relation to operational equipment, and in the 
value of repeated use of the trainer. Because of the way 
the trainer was being used, no meaningful scores were obtained 
on trainer reliability or on comparison with similar trainers. 
In the former case, only one respondent provided any ratings 
and it was felt that, in its present mode of use, valid 
evidence of reliability was lacking. In the latter case, 
there were really no similar trainers with which the 2H87 
ecu Id be compared. 

It was evident that in the opinion of the users the 
trainer has major deficiencies both as a flight trainer and 
with respect to the visual presentation of the simulated 
environment. In an effort to understand the nature of these 
felt deficiencies, the foil owing comments from instructor 
and administrative personnel are presented, 

"The concept of this device was excellent but it 
exceeded the state-of-the-art at the time of 
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procurement • Rapid turnover of project officers , 
infinite and ambi guous specifications, improper 
progress evaluations , misconception of the purpos e 
of the device, and political sparring created 
chaotic results," 

"Device acceptance to this day is shrouded with 
misunderstanding. The result was that an un- 
suitable device was procured at an exorbitant 
cost, and it could not be used for student train- 
ing. It lay idle for a year. After a suggestion 
that the device be either utilized or turned over 
as excess, the present concept of use was devel- 
oped. It is felt however that the device will 
soon be lost since it is a poor substitute for its 
original concept," 

"The trainee cannot manually control this trainer 
due to differences between performance of the 
trainer and the aircraft. It doesn't fly like 
the aircraft , " 

"Instructors cannot handle the trainer. It would 
do more damage than good to the student in its 
present state. The instructors are not qualified 
because they are trainer operators not trainer 
aviators. Qualified LSO's should serve as in- 
structors or at least oversee the flight," 

"Everyone who has flown the trainer has an un- 
favorable opinion of it. It is too difficult to 
control, and the presentation is not good enough," 

"At present the trainer is of limited use for 
students because of its poor response or feel. 
The only value for students is to show them lim- 
ited approaches and catapult shots. The LSO 
instructors feel the trainer would give the stu- 
dent a feeling of insecurity if they did poorly 
in flying the trainer. This would (have) an un- 
desirable effect on their level of confidence 
when it is time to go to the ship," 

"Instructors are not well qualified to evaluate 
student performance in the trainer because they 
are not LSOs , Therefore they can only evaluate 
the computer readout which is not the same as 
the LSOs' analysis of each approach," 

"Student personnel who have flown the 2H87 feel 
that the inputs generated by the student are 

95: u 
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over control led by the computer inputs, thereby 
giving a false and unrealistic picture of an 
actual carrier approach. They also feel that 
visual resolution is very poor," 

"I would like, for once, to see the Navy utilize 
the opinions of the user and not rely so heavily 
on engineers and computer specialists who have 
never seen a carrier approach and landing in 
developing a trainer such as this one. If it 
would have been properly engineered, this trainer 
could have been a tremendous aid in training 
students • " 

"As far back as 1967, VT4 uc pilots to the con- 
tractor's plants to help the engineers work out 
the problem areas. My pilots reported to me when 
they returned from seeing the mock-up that such 
information as glide slope and so on were all 
wrong. This problem was a consequence of not 
having an LSO , who knew what these parameters 
should be, serving as a project manager. It was 
no fault of the contractor; they were only going 
on what specifications were given to them by 
NTDC," 

"A new high resolution camera that is now avail- 
able through advancement of the state-of-the-art 
would increase the use of the trainer 99%, Re- 
programming the trainer to fly in accordance 
with the aircraft is a relatively simple matter 
I am told. The trainer would be extremely 
valuable to this command if it worked as ad- 
vertised, " 

In summary, the whole history of this device is one of 
confusion and concern, NTDC developed the specifications for 
a prototype device with the apparent objective of making it 
versatil'i enough to evaluate a variety of visual systems 
that mi^.*t be utilized. In fact, it was built with a rather 
inflexible visual system that is more complex than necessary. 
In many respects the engineering of the trainer seemed to 
be very good and the concept certainly was without fault. 
However, the failure to adequately simulate the performance 
of the T2 aircraft seems to have been an overwhelming in- 
fluence that precluded trainer acceptance for any purpose. 

Nevertheless, an interesting feeling exists concerning 
the value of the 2H87, With all its admitted faults, some 
people feel that the programmed visual catapult shots have 
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a positive training value. "Students can at least be given 
a presentation of how rapidly things change during catapult 
takeoffs." This feeling is related to an incident on a car- 
rier where a student pilot was involved in a bridle breaking 
during run-up. When the bridle parted, he thought the cata- 
pult had been fired and let the airplane roll. Subsequently 
he went over the side and was killed. Everyone associated 
with the trainer relates this story as an example of how 
effective the trainer could be in at least acquainting the 
student with the visual changes associated with catapult 
operat i ons . 

Because of the nature and small number of qualified 
respondents, the relationship between operational experience 
and trainer acceptance cannot be described with confidence. 
It is doubtful whether any significant relationship could be 
shown in the absence of student pilot opinion. However, the 
data are shown in Table 14 in the interest of completeness. 
Experienced personnel tended to be particularly critical of 
the environmental simulation, vehicle performance, and 
control effects . 

TABLE 14. AVERAGE RATINGS OF THE 2H87 
BY LEVEL OF EXPERIENCE OF USERS 



Experience Level 





Low 
(N=2) 


Medi um 
(N=l) 


High 
(N-5) 


Average Ra ti ng, 
Simul ati on Factors 


5.6 


6.6 


5. 1 


Average Rating, 
Training and 
Situati onal Factors 


4.5 


10.2 


3.4 



PROFILES OF THE REMAINING TRAINERS 

The remaining trainers in the sample were assigned mid- 
range values on the composite criterion of acceptance, 
representing neither outstanding examples of acceptance nor 
rejection. They did differ substantially in acceptance 
with respect to particular factors, however. Their profiles 
will be presented in the sections that follow roughly in 
the order of the level of acceptance. 

DEVICE 14B40: RADAR/MAD OPERATOR TRAINER. The 14B40 was 
designed to train sensor operators in the problems and pro- 
cedures associated with non-acoustic ASW equipments, specif- 
ically radar and MAD equipment. It is a well accepted 
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trainer with a composite criterion score of +3.3. As indi- 
cated by Figure 22, this trainer received quite high average 
ratings on the simulation factors; it was rated above average 
in display of information, equipment performance, control 
effects, sensor performance, and target reactions. However, 
the ratings on these factors were somewhat below the judged 
importance level . 

With respect to training and situational factors, the 
14B40 generally rated about the average of all trainers 
(Figure 23). However, it was rated very high in problem 
setup and above average in training effectiveness in com- 
parison to operational equipment and to other similar trainers. 

In general, the instructors felt that the 14B40 is a 
very effective training device with a few minor improvements 
needed such as ''more realistic weather effects and radar 
range cutoff points due to altitude.*' The first class of 
students trained with the 14B40 (it is a new trainer) "im- 
pressed the squadron people with the amount of information 
they carried back with them." 

There was some early concern on the part of maintenance 
people about the reliability of the trainer that may be 
reflected in the below average rating on reliability. These 
fears have failed to materialize according to the personnel 
interviewed. Part of the problem was that a large number of 
changes to the trainer have kept maintenance personnel un- 
certain concerning whether schematics , manuals , and other 
technical information are up-to-date. Availability of parts 
has been a problem, especially replacement parts for GFE 
items. There reportedly have been major problems with the 
tactical situation display. 

"The factory school for this trainer was struc- 
tured for computer programming with no real help 
being given for practical maintenance of the 
systems. Preliminary manuals are still in use, 
having not been replaced by the final manuals by 
the contractor . " 

There was a hint of slightly greater acceptance of the 
14B40 among personnel who had had considerable operational 
experience with radar and MAD systems (see Table 15). Ex- 
perienced personnel rated the trainer noticeably higher on 
simulation of sensor performance than neophyte personnel 
but rated it lower on reliability and level of training. 
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TABLE 15. AVERAGE RATINGS THE 14B40 
BY LEVEL OF EXPERIENCE OF USERS 



Experience Level 





Low 
(N = 5) 


Medium 
(N = 6) 


High 
(N=6) 


Average Rating, 
Simul ati on Factors 


9.5 


11.7 


11.3 


Average Rating, 

Trai ni ng and 

Si tuati onal Factors 


10.9 


9.6 


1 1 . 1 



DEVICE 2F64A: SH3D HELICOPTER CREW TRAINER, The 2F64A is a 
weapon systems trainer used to train SH3D helicopter crews 
in I CS procedures , dipping procedures , crew coordination, 
knobology, and practice in tracking. It received a somewhat 
above average composite criterion score of +2,0, 

The 2F64A prof i le on s imulation factors is shown in 
Figure 24. It is apparent that the trainer approximated the 
the average of all trainers as a group on most factors al- 
though it was rated somewhat above average in display simula- 
tion, sensor performance, and target reactions. However, it 
failed to reach the judged level of importance in these 
areas and also shows an "importance differential" on simula- 
tion of vehicle performance and control effects. 

With respect to training effectiveness (Figure 25), the 
trainer was also rated about average although it received a 
superior rating on immediacy of performance evaluation and 
somewhat above average scores on value of trainer time and 
comparison with similar trainers . 

During the interviews complaints were recorded concern- 
ing the realism of the sonar displays and limited simulation 
of the effects of water conditions and their influence on 
target classification. The trainer was criticized for having 
no capability for raising or lowering the dome, and poor ICS 
communicati ons . 

A small percentage of pilots have an unfavorable opinion 
of the device because they feel it does not fly like a heli- 
copter . Administrative and instructor personnel reportedly 
try to combat this viewpoint by pointing out that the trainer 
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is primarily for procedures training. The slow pace of up- 
dating the trainer to match the performance characteristics 
and other features of the actual aircraft have created some 
negative opinions . 

"Pilot trainees feel that the instructors are 
familiar with trainee reactions and the average 
ability of students but they are not able to 
judge the ability of pilots accurately since 
they are not pilots themselves. Pilots feel 
that the trainer is excellent for system NATOPS 
f ami liarization and cockpit organi zation and 
understanding. However they do not like to be 
evaluated by enlisted personnel." 

"The flight characteristics leave a lot to be 
desired. There is too much instrument lag in a 
few instruments. For instance^ the remote mag- 
netic indicator and the vertical gyro indicator. 
There is no rudder stick balance with the needle 
ball display. Gyro movement is not realistic. 
The trainer will not simulate hovering." 

The sonar equipment is reportedly a persistent mainte- 
nance problem. 

"Due to limited use in the past, the system has 
not been kept up to snuff. Now that utilization 
has increased, increased maintenance is required. 
Documentation is poor and we must proceed by 
trial and error." 

"The GFE sonar equipment presents maintenance 
problems . Signal input levels are so low that 
infinitesimal change will foul the whole display." 

"The reputation of the trainer has improved con- 
siderably since its introduction (at Quonset 
Point) . The primary fault of a digital computer 
system is that a down computer means a down 
trainer. Maintenance people did not know enough 
about the digital equipment when the trainer was 
new and sometimes a Chinese fire drill would 
ensue if a computer quit. The trouble in some 
instances was external to the computer. The 
reputation that the trainer has had for being 
bad (unreliable) is not necessarily a valid one." 

As indicated in Table 16, there were no apparent differ- 
ences in overall acceptance of the 2F64A as a function of 
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user experience in the SH3D, However, experienced personnel 
rated it significantly lower on training software and com- 
pleteness of performance evaluat i on • 

TABLE 16. AVERAGE RATINGS OF THE 2F64A 
^ BY LEVEL OF EXPERIENCE OF USERS 



Experience Level 





Low 
(N=ll) 


Med1 urn 
(N = 7) 


Hi oh 
(N-8) 


Average Rating, 
Simulati on Factors 


10.7 


9.7 


1 0. 1 


Average Rating, 

Training and 

Si tuati onal Factors 


10.9 


10.0 


9.5 



DEVICE 2F66A--A COMPARISON OF TWO INSTALLATIONS OF THE SAME 
TRAINER. The 2F66A was developed to provide pilot and crew 
training for the S2E ASW aircraft. Specific emphasis was 
placed on pilot emergency and instrument flight procedures 
and on the tactical interaction between all members of the 
crew. Installations at both San Diego (North Island) and 
Quonset Point were studied for possible differential accep- 
tance of the OFT portion. Ratings were obtained on the 
entire trainer at San Diego but only the OFT portion at 
Quonset Point. 

As indicated in Table 4, neither trainer departed mark- 
edly from the overall average with respect to the composite 
criterion of acceptance. However, the San Diego trainer, 
with a score of +1.1, was rated about three units higher than 
the Quonset Point trainer with a score of -2.2. This was due 
in part to the higher utilization at San Diego but also to 
the fact that a greater proportion of the raters at Quonset 
Point were pilots. It was evident that the pilots at both 
locations rated the 2F66A lower than did the tactical crew 
members because of what they felt were serious deficiencies 
in its flying characteristics. 

A comparison of Figures 26 and 27 indicates that the 
Quonset trainer was rated particularly low on vehicle per- 
formance and effects of controls. A check of the data 
showed that San Diego pilots rated that trainer as similarly 
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deficient in those areas. However the higher ratings given 
the San Diego trainer by tactics personnel obscured the sim- 
ilarities. This suggests that in future studies the opinions 
of pilot and tactical personnel should be treated separately 
in analyzing the data. Unfortunately, in the present study, 
it was not possible to include enough personnel in each cate- 
gory to get reliable results for different sections of the 



The other differences in the simulation profiles of the 
two trainers can similarly be explained. The San Diego 
trainer was rated somewhat lower on information displays and 
sensor performance. This probably reflects a more critical 
evaluation of those characteristics of the trainer by tac- 
tical personne 1 . 

The profiles of the two trainers on training effective- 
ness factors are very similar (Figures 28 and 29), Both 
received average ratings in most respects although both were 
rated down significantly in the area of software. The reason 
for the unusually high rating of the Quonset trainer on level 
of training is not known. It may again reflect the inadequate 
sampling of tactical personnel at the Quonset installation. 

Some typical comments concerning the 2F66A at Quonset 
Point follow. 

"The trainer affords good ASW tactics practice, 
emergency procedures practice ,* and practice in 
engine management during unusual circumstances.'' 

••Some instructors are not instructors at all. 
They merely turn on the machine and push buttons 
when requested to do so. It would be beneficial 
if they were qualified to instruct. They know 
nothing about ASW, flying, etc." 

"Learning to fly the trainer creates some possible 
negative training because of the difference in the 
flying techniques that must be used in mastering 
t;he trainer as opposed to those used for mastering 
an aircraft. Having to concentrate on the handling 
ability of the trainer detracts from the purpose of 
training . " 

"I believe this trainer is the best way to train 
a VS crew. All aspects of the VS mission can be 
accomplished in the trainer short of an actual cat 
shot and arrested landing, excluding searchlight 
and camera pod hops. lif you do all that is expected 



trainer. 




NAVTRADEVCEN 70-C-0276-1 




a u 0 3 s 



109 



NAVTRADEVCEN 70-C-0276-1 



of you in the trainer, I believe you could fly 
an actual mission as far as ASW goes. It is a 
valuable training aid to say the least." 

"Aircrewmen would like to see spurious contacts 
for the tactics portion; there are few environ- 
mental qualities available in the tactics dis- 
plays such as washover, sea return, and ambient 
nois e . " 

The following are selected comments about the 2F66A 
made by San Diego personnel. 

"The trainer is very effective if properly uti- 
lized. It simulates real conditions so well 
that in some instances frustrations have been 
induced just as if the equipment being used were 
real 

"Effective training is hindered by instructor 
capability. When they don't know enough about 
the trainer to simulate real world problems, 
the training suffers." 

"Instructor techniques seem not to be standardized. 
Newer instructors are not as good as the older 
ones. A lot of time is spent in stopping the prob- 
lems and resetting them." 

"The device trains very well in 'knobology,* basic 
operator functions , instrument flight char act er- 
istics for the pilot, and ASW plotting. It is, 
however, limited in the scope of simulation." 

"The trainer is being used to train skills other 
than those originally intended by the designer. 
It is being used in expanded tactics. The trainer 
is versatile enough so that the individuals who 
want to spend time with it can learn of new ways 
to incorporate it into the deve lopment of their 
tactical skills . " 

"One of the major deficiencies of the trainer is 
that the equipment does not keep up with the 
actual aircraft; therefore some negative training 
results . Pilots in particular are sensitive to 
thes e differences . " 

"The trainer is too perfect. It doesn't provide 
for the gray areas. It doesn't teach what to 
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expect in the real world. The signals are ideal; 
targets are easily detected." 

"There is a lack of realism in the LOFAR and CODAR 
displays. They are much too easy to d^al with. 
Some of the systems in the trainer are outdated 
such as the JULIE and gram analysis capability 
with the JEZ system. Students feel that tactics 
and conditions are changing faster than it is pos- 
sible to update the training equipment." 

"Schematics are very poor and a high degree of 
technical back ground is necessary to use the ones 
that are provi ded . The technical manual is no 
good and is not used. Drawings and schematics 
are lousy. They are good for general familiariza- 
tion but are not reliable for any detailed main- 
tenance work . " 

"Maintenance personnel had to set up their own 
school with regard to learning the computer and 
how it worked. The information provided by NTDC 
was much too skimpy." 

"Government furnished equipment is generally poor 
and extremely hard to maintain. An example on 
this device is the ASN-30 plotter. The Packard- 
Bell 250 computer is (also) a problem. It is a 
general purpose computer but repair parts are 
scarce and very slow in being supplied." 

"As an OFT, it (2F66A) is of value only to the 
new student to learn systems and s ome procedures . 
Visually it is an exact duplication of the air- 
craft, but control pressure and responses are not 
at all like the aircraft.". 

"As a WST, it is of value for learning crew co- 
ordination and procedures and for solving complex 
problems. It is of great value for improving 
readiness; if all training was to be obtained in 
the aircraft the time and expense would be 
prohibitive. " 

"I would also comment on the amount of down time. 
It can be infuriating to be near a solution of a 
long and difficult problem and have the computer 
dump . " 

"During a training session when a whole crew was 
working on an exercise to obtain a qual, the 
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computer dumps the problem destroying the time 
limit involved and the procedure necessary for a 
qual • " 

The results of this accept an ce analysis of the 2F66A 
trainers appear to be entirely compatible with the mch more 
detailed evaluation of the training effectiveness of the de- 
vice recently completed by the Bunker-Ramo Corporation (12), 
In that study, Meister et al. concluded that the 2F66A was 
effective for training all crew members but that it was more 
effective for beginning students than for operational and 
reserve personnel. The trainer received particularly low 
ratings on realism of the control forces, and realism of 
information displayed to the sensor operators and "tfuly-^a^rer.- 
age scores on "overall effectiveness." 

While the instructors considered the trainer effective, 
problems of equipment malfunctions and insufficient fidelity 
"have reduced confidence and lowered the overall accept- 
ability of the device as an integral part o^ air ASW training." 
It was further concluded that important improvements in 
training for all S-2E personnel could be effected through im- 
proved systematic utilization of the existing capabilities* 
of the training device. 

It was suggested that among ihe steps that could be 
taken to improve trainer effectiveness were "(1) more sys- 
tematic variation of the nature of the training sessions; 
(2) increased trainer usage; (3) more systematic utilization 
of the trainer in both individual position and team training 
modes; and (4) improved trainer maintenance, enhanced fidel- 
ity, and increased trainer hardware capability." 

Table 17 suggests that these trainers are somewhat less 
accepted by the more experienced personnel. Both trainers 
were rated particularly low by experienced personnel on 
software. In addition, the Quonset trainer was rated very 
low on sensor simulation and the San Diego trainer was rated 
particularly low on control effects. 
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TABLE 17. AVERAGE RATINGS OF THE 2F66A 
BY LEVEL OF EXPERIENCE OF USERS 



Experience Level 



2F66A 
(Quonset Point) 


Low 
{N=2) 


Medi um 
(N = 4) 


High 
(N=4) 


Average Rating, 
Simulation Factors 


9.2 


9.5 


9,6 


Average Rating, 
Training and 
Situational Factors 


12.6 


9.6 


6.8 


2F66A 
(San Diego) 


Low 
(N=8) 


Medi um 
(N = 8) 


High 
(N = 8) 


Average Rating, 
Simulation Factors 


1 1 .0 


!0.5 


9.6 


Average Rating, 
Training and 
Situational Factors 


10.4 


1 0. 1 


8.4 



DEVICE 14A6A: ASW COORDINATED TACTICS TRAINER, The objective 
of the^l4A6A was to provide training in coordinated ASW 
operati'-ons varying from a relatively simple situation involv- 
ing a f\)vt ASW units, to complex exercises involving an entire 
ASW tasii.^group (up to 250 key people) . The trainer was to be 
utilized to conduct demonstrations, provide for coordinated 
training, and evaluate tactics. It was designed to illus- 
trate: (1) the capabilities and limitations of surface ships, 
submarines, aircraft, helicopters, and ASW task groups; (2) 
the utilization of search, localization and attack equipments 
and weapons; (3) the control of multi-unit forces in conduct- 
ing search, localization and attack; (4) communication, 
plotting and recording facilities and procedures for the 
rapid exchange and analysis of necessary information; (5) com- 
mand relationships and responsibi lities; (6) realistic enemy 
submarine threats; and (7) offensive and defensive missions • 

The trainer was to provide training for decision-making 
personnel in coordinated , inter -type, ASW tactical situations. 
The participants include aviation, surface, and submarine 
personnel from enlisted through flag rank. In general, it was 
expected that the participants would be "moderately to ex- 
^ pertly skilled in their particular individual type functions; 
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however, they may be found to require practice, to varying 
degrees, in applying coordinated inter-type ASW tactics." 

In terms of the composite acceptance criterion (Table 
4), the 14A6A trainer fell almost in the middle of the 
distribution on the acceptance scale with a score of +0.6. 
Its level of acceptance can clearly be described as "modest. 
The reasons for this are probably most evident from the 
profile on simulation factors (Figure 30) . It is clear that 
the 14A6A was rated below the average of all trainers on all 
simulation factors. It was rated particularly low on simula- 
tion of controls, displays, sensor performance, target 
reactions, and weapons firing and control. It is interesting 
to note, from the ratings of importance on these factors, 
that the participants did not regard high fidelity simulation 
of sensor performance, weapons and displays as important for 
this trainer as it was for individual ASW weapons systems 
trainers (for example, the 14A2) . Nevertheless, the importance 
ratings were considerably higher than the values assigned to 
the 14A6A in many important respects. 

In the contrast to the ratings on simulation factors, 
the 14A6A was rated considerably more favorably with respect 
to general training effectiveness factors (see Figure 31) . 
Dissatisfaction is evident, however, with respect to the 
completeness and immediacy of performance evaluation. It 
received above average ratings in reliability and level of 
training. With respect to most other characteristics, it 
did not depart in any pronounced way from the average of all 
trainers . 

The failure of the 14A6A to meet the levels judged as 
important on various simulation factors perhaps reflects a 
frequently mentioned criticism of the trainer, namely, that 
considerable time and effort must be expended learning to 
operate the trainer per' se. The equipment in the trainer is 
"so unlike operational equipment that it is necessary to 
spend too much time" learning its peculiarities. 

A possibly significant deficiency is that since an entire 
ship's team occupies a single small space, many problems in 
internal communications can be avoided by direct face-to-face 
conversation, or by looking at a plot or display not normally 
available. In addition, the display presents unambiguous 
information whi ch makes the prob lems of communication un- 
realistic. The highest rating received by the 14A6A was in 
the area of communications procedures. This supports the 
contention of mai\y that the 14A6A is primarily a communications 
procedures trainer . Nevertheless , internal communications 
are poorly simulated. 
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With respect to controls, the 14A6A equipment is very 
unlike operational equipment. The raters accept this in 
principle, but nevertheless think it should be more realistic 
than it is . 

The sensors in the 14A6A are very unlike the real sen- 
sors and they, again, provide unambiguous information. With 
respect to target reactions, the trainees "know that the 
instructors are playing games." One commanding officer com- 
mented that he very seldom heard any discussion of how foreign 
submarines are known or suspected to act in the various tac- 
tical situations presented in the trainer. 

With respect to weayon control and firing, it is em- 
phasized that the 14A6A was designed to carry the exercise 
up to the point of firing only. Evidently the assumption 
was that the firing phase of training would be taken care of 
in other trainers. The very low rating on this factor clearly 
indicates the opinion of many user personnel that this is not 
good enough. Several trainees and a few instructors com- 
mented that the ideal trainer would be one with the general 
capability of the 14A6A but composed of several 14A2 mock-ups 
with their much more detailed capabilities. 

The comparatively low ratings on performance evaluation 
probably reflect the fact that there is no provision for im- 
mediate feedback at critical points in the exercise. Exer- 
cises typically last a full half day with problem reconstruc- 
tion on a large screen at the end. During large scale 
exercises, enlisted personnel often take notes at individual 
stations; however, there is little in the way of critiques 
of individual ship's actions. What there is tends to take 
the form of "you should have done thus and so" rather than 
a diagnostic determination of why a particular action was or 
was not taken by a given unit. The performance of individual 
operators is not critiqued, of course, because they are not 
utilizing actual operational gear. An exception to these 
observations is the evaluation of the overall communication 
plan that is established for the purposes of the exercise. 
This is evaluated rather thoroughly. 

The reliability of the trainer on the whole was rated 
very high. However, it is not unusual to have crews shift 
from one mock-up to another because of an equipment casualty. 
Since the trainer is rarely used to full capacity, this does 
not result in a significant loss of training time. The tech- 
nicians complain, however, that the 14A6A was built using a 
line of parts left over from other projects. They reportedly 
often discover that the part they require is no longer in the 
Navy supply system or, worse, was built by a company no longer 
in business. 
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It seems likely that the ratings or the 14A6A would have 
been much higher in the absence of some of the simulation 
shortcomings mentioned earlier. However, this is little 
doubt that the trainer helps to recognize and correct many 
procedural problems before the units go to sea, particularly 
when used prior to a major fleet exercise, "Everyone would 
prefer more at sea training, but since that is not possible, 
the A6 is a better- than-nothing substitute," 

There is evidence in the data (Table 18) that the sim- 
ulation deficiencies are more strongly felt by personnel 
with the greater experience • This was true particularly 
with respect to communication problems , displayed informa- 
tion, and weapon firing effects, 

TABLE 18. AVERAGE RATINGS OF THE 14A6A 
BY LEVEL OF EXPERIENCE OF USERS 



Experience Level 





Low 
(N=3) 


Medi um 
(N=ll) 


High 
(N=15) 


Average Rating, 
Simulation Factors 


9. 1 


7 . 5 


6.7 


Average Rating, 
Training and 
Situational Factors 


8.8 


1 0 . 1 


9.3 



DEVICE 2B21: T-28 INSTRUMENT TRAINER. The 2B21 is an in- 
strument flight trainer with multiple cockpit installations • 
It is designed for basic training in the (1) operation and 
use of individual instruments; (2) development of an in- 
strument-scan pattern; (3) coordination of controls and 
instruments in basic flight patterns; (4) use of radio navi- 
gation facilities, i.e., VOR/TACAN, and ADF and ILS; (5) 
GCA-landing practice; (6) voice procedures, where applicable 
in the above areas of training; and (7) operational flight 
training of the T-28B aircraft. 

The trainer was to simulate "normal aircraft operation 
with respect to flight and engine performance, flying 
qualities, aircraft systems operations, radio communication 
and navigation systems operation, and flight path." The 
simulation was to result in appropriate response on the part 
of the trainer's indi cators , control reactions , and the flight 

(. 
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path recorder. The trainees utilizing the trainer were 
presumed to have no prior experience in instrument flight 
and navigation, and only limited flight instruction. 

The composite criterion score for the 2B21 Was +1.4, 
just slightly above the average of all trainers. It is 
evident that the trainer is accepted but not without some 
reservation. The reasons may be reflected in the profile 
on simulation factors (Figure 32) . The trainer was rated 
well below average in simulation of vehicle performance and 
control effects. In relation to judged importance, these 
deficiencies are particularly severe. In contrast, the 
trainer was rated as distinctly above average on information 
displays and sensor performance. 

Tht> 2B21 was rated above average on several important 
training effectiveness factors. These include problem set- 
up, software, performance evaluation, reliability, and 
value of time in the trainer. However, the trainer was 
rated below average on level of training, comprehensiveness 
of training, and on use of the trainer versus operational 
equipment. It was also rated somewhat below average in com- 
parison with other trainers of this type. (See Figure 33.) 

Some possible reasons for the mixed attitudes of accep- 
tance and rejection are reflected in the following comments. 

"The trainer does not fly like the real aircraft. 
It's a poor basic air trainer but a very good 
scan procedures, basic instrument, and radio work 
instrument trainer. Without it, students' progress 
would be s low. " 

"Some instructors feel that the device hurts scan 
and some basic procedures and also basic air work 
for trim, rudders, etc. Making the trainer re- 
semble the T28 aircraft may have been a mistake. 
Students expect the trainer to respond like the 
aircraft and it was not designed to do that. 
Lack of realism in the control responses of the 
trainer is a continuing problem for the instructors." 

"Rudder pedals in the trainer are oversensitive. 
Students transmit information to each other to the 
effect that they should fly the trainer with their 
feet on the floor. Some negative training occurs 
becaus e of this condition . They forget or have 
negative carry-over and try to fly the aircraft in 
the same way." 
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"The turn and bank indicator does not react to 
the application of power. Control sticks are 
not smooth, seem to bind, and cause the trainee 
to over- con tro 1 the trainer." 

"No status accrues to the student as a result of 
flying the trainer. They are still playing the 
"hours game** for status and therefore time in the 
trainer only detracts from the time they have 
available to accumulate hours in the air." 

"The more intelligent students appreciate the in- 
tent of the trainer and benefit more from the 
training time. They recognize how the trainer 
satisfies training needs in the area of instrument 
scan procedures, and voice reports, but they 
recognize also that it is certainly no good for 
basic air work. Flying the "Link," as they refer 
to it, is not at all like flying the aircraft. 
Some students feel that flying a Link can be 
detrimental to flying in the aircraft because of 
great differences in control pressure and trim." 

It is noteworthy that student responses to the question- 
naires in this study were very similar. They expressed 
disappointment that they could not increase their airmanship 
proficiency in the trainer and were concerned by the un- 
reliability of the internal communications system between 
instructor and student. The trim and rudder problems were 
very distracting to them. Since this was their first ex- 
posure to air trainers, it is possible that the deficiencies 
noted will have a lasting effect on the opinion of some 
students with regard to air trainers in general. 

There were no large differences in acceptance of the 
2B21 as a function of level of experience (see Table 19). 
However it is interesting that the more experienced personn 1 
rated the trainer higher on level of training and value of 
time in the trainer than did those with low experience. On 
the other hand, they rated it lower in comparison with similar 
trainers . 
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TABLE 19. AVERAGE RATINGS OF THE 2B21 
BY LEVEL OF EXPERIENCE OF USERS 



Experience Levpi 





Low 

{N=18) 


Medl um 
(N = 3) 


High 
(N=6) 


Average Rating, 
Simulation Factors 


9.3 


9.9 


9.6 


Average Rating, 

Trai ni ng and 

Si tuati onal Factors 


9.7 


9.6 


10.8 



DEVICE 2F65: E-2B WEAPONS SYSTEMS TRAINER. The 2F6S was 
designed to simulate the tactical mission of the E-2B 
weapons system. The E-2B is an all-weather airborne early 
warning and control airplane. It has a tactical crew con- 
sisting of an ECM operator (copilot) , CIC officer, air con- 
trol officer, and radar officer who are responsible for the 
operation and control of all avionics equipment. The co- 
pilot doubles as ECM operator when the aircraft is on station 
and provides ECM inputs to the avionics system computers. 

The tactics system of the E-2B performs the following 
functions: detection, acquisition, automatic tracking and 
identification of targets, report of target positions, and 
control of interceptors. 

The device was intended for the use of carrier based AEW 
squadrons, to train fleet pilots and AEW/CIC crews under the 
direction of qualified instructors. The device was to pro- 
vide training in all of the necessary cockpit techniques , 
flight planning, emergency procedures, instrument flight pro- 
cedures, and tactical procedures associated with the weapons 
system. 

On the composite criterion (Table 4), the 2F6S received 
a somewhat below average overall rating* (-2.2). In the pro- 
file on simulation factors (Figure 34), it is evident that 
deficiencies were felt in the areas of vehicle and equipment 
performance, control effects, communi cat i on procedures , and 
information displays. In addition, the trainer was rated 
distinctly below the importance ratings with respect to sen- 
sor performance , target reactions , and weapons control • 
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With respect to training effectiveness factors, the 
device fared much better (see Figure 35). The trainer was 
generally rated average or better in all respects with the 
exception of software. Despite any of the felt deficiencies 
already mentioned, the value of time spent in the trainer 
was rated as above average. 

It was clear to the study team that the 2F65 had two 
distinct levels of acceptance. The OFT portion was generally 
regarded not acceptable because of the problems associated 
with simulated flight, particularly aircraft control and 
response. Evidently the analogue simulation of these func- 
tions has proven very unsatisfactory, and changes in aircraft 
performance parameters are not easily incorporated in the 
trainer. Reportedly, "many pilots lose sight of the effec- 
tiveness of the trainer for emergency and procedural training 
because of these deficiencies," 

The tactics section of the trainer has a much higher 
level of acceptance than the OFT, This appears to be because 
it can be used to develop problem solving ability and perfor- 
mance skills by NFO's, Acceptance is not unqualified, how- 
ever, NFO's feel they would derive more from the training 
sessions if "greater realism at different levels of complexity" 
could be incorporated into the video displays. 

Squadron personnel are not always as clear as they might 
be concerning the potential value of the trainer and the 
training personnel are not considered fully responsive to 
their training needs. No one seems completely happy with the 
trainer. Major updates have been made and are ongoing but 
the consensus seems to be that the effort is futile. Because 
of original inadequacies, it is felt that little can be 
accomplished to make the 2F65 into an entirely acceptable 
training aid. Early unrewarding experiences seem to have 
carried over and projected into future expectations. 

Some verbatim responses from the questionnaires follow, 

"This trainer satisfies needs for training in voice 
reporting and manual tracking; exposes students to 
data link operations; acts as an interface between 
the classroom and the real world; and provides a 
base for performing and experimenting with tactics," 

"The trainer does not realistically simulate video 
modes , no side lobes, etc. Radar interpret at ion 
is la eking and communi cat i ons net background noise 
is also lacking," 
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"More tactics problems are needed, such as carrier 
controlled approaches, strike controlled fleet 
exercises, and war games.'' 

"Radar and IFF simulation are not worthwhile," 

"It is felt that the trainer could benefit from a 
Navy man as the primary coordinator for all 
activities related to the trainer,,. a managing 
director who can better relate the use of the 
trainer to the operational world, a real Navy 
NF0/E2B type with sufficient expertise to ramrod 
this show and keep on top--this would be his 
primary duty . " 

"Instructors at some activities do not thoroughly 
know the operations and limitations of airborne 
equipment. Some instructors have not been checked 
out in operation of the trainer when they are 
assigned to it. Occasionally both of these con- 
ditions apply to the same instructor." 

"The operators and other people associated with 
the trainer have no general flight experience in 
the type of aircraft they are trying to simulate. 
It would be invaluable to have these people ex- 
perience familiarization flights in the aircraft 
itself," 

"More rapid, frequent and direct contact is re- 
quired with project personnel both at NTDC and 
the Naval Air Systems Command, These support 
personnel have the most current information on 
trainer status, use and need and are at the end 
of the chain of decision. They are the last to 
know concerning the many items affecting the 
ability of the trainer to meet its mission. There 
is no wish on the part of the lower people to 
usurp power; they would just like to be provided 
with a method for making valid inputs for trainer 
changes and improvements and they would like to 
be informed of what the training requirements are 
going to be , " 

"Problems occur in troubleshooting because of the 
way the schematics and other parts of the manuals 
are laid out and these cause considerable confusion 
and wasted time. The trainer has undergone ex- 
tensive updating which has required a lot of revision 
of the technical manuals. Schematics and drawings 
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are for the most part insufficient. Some are too 
brief or vague to be of much help; some are so 
full of errors they are useless • Technical man- 
uals appear to be written by people who take an 
awful lot for granted, for example, engineers. 
Parts of some systems are scattered over many 
sections of the manuals," 

"Training manuals are not used regularly because 
the individuals are familiar enough with the 
equipment • Write-ups in the manual are weak and 
don't necessarily relate to the training equip- 
ment. Scuttlebutt has it that the documentation 
was subcontracted and deve loped from engineering 
notes . " 

"There needs to be a more responsive method for 
getting aircraft changes incorporated into the 
trainer quickly. The decision to include air- 
craft changes (in the trainer) are too far re- 
moved (in time) from operational (changes). 
Negative training occurs because of this." 

Some observations from trainees : 



"The trainer is good for cockpit familiarization, 
normal pre-starts, starts, takeoff and landing 
procedures, and eye scan development. The students 
are generally left on their own. Some come to the 
trainer very well prepared; others come with a 
lackadaisical attitude, sometimes fostered by pre- 
vious troubles in the trainer. The students feel 
that the trainer doesn't simulate conditions well 
enough to do much training. This applies to the 
OFT portion. The opinion is that the trainer bears 
no resemblance to the E2 aircraft. It is good for 
emergency procedures within limitations and it 
makes a good cockpit familiarization trainer. But 
it doesn't fly too well and the navigational aids 
don't work. Very frequently it fails during flight." 

"Many instruments are too imprecise to give accurate 
simulation . Many programs were not included or were 
erroneously installed and repr ogramming is difficult 
and expensive. Flu.ght curves, etc., would need re- 
scaling to make them match the per f ormance of th e 
aircraft. The original design information was in- 
accurate and since these features are hard-wired 
into the trainer, there is little opportunity for 
change . " 
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"It is felt that direct liaison between squadron 
NATOPS people and NTDC is necessary. At present 
many inputs (suggestions for change) are received 
at widely separated periods of time. What is 
needed is continuous liaison; also, most necessary 
changes are beyond the capability of NTDC repre- 
sentatives. Much effort was expended by NTDC to 
fix the OFT portion of the trainer. But most of 
what was initiated was inconsequential insofar as 
training was concerned. The present trainer is 
beyond the scope of changes necessary to bring it 
up to date." 

One respondent had had experience previously with a com- 
mercial airlines simulator. He felt the difference was 
startling between the commercial simulator and the 2F65 and 
that the Navy could benefit by incorporating some of the 
features used in commercial simulators. Quite a few trainees 
were aware of what the airlines were doing with simulators. 
They felt that trainers can be made to do the job but that 
this one was not. Their question was, "Why?" 

The more experienced personnel were definitely less 
accepting of the 2F65 than neophyte personnel (see Table 20). 
This was particularly true in the areas of vehicle perfor- 
mance, communication problems, displayed information, problem 
setup, training software, completeness of performance evalua- 
tion, level of training, and value of trainer vs. operational 
experience , 

TABLE 20. AVERAGE RATINGS OF THE 2F65 
BY LEVEL OF EXPERIENCE OF USERS 



Experience Level 





Low 
(N = 3) 


Medi um 
(N=5) 


High 
(N=6) 


Average Rating, 
Simulation Factors 


10.9 


8.5 


8.4 


Average Rating, 
Training and 
Situational Factors 


11.8 


10.4 


7.6 
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SECTION VI 

A POST HOC APPRAISAL OF THE RATING SCALES 

One of the objectives of this study was to develop a 
methodology which could be used in the future by NTDC to 
assess the acceptance of training devices. It is felt that 
the acceptance profiles presented in the preceding section 
are clearly useful and diagnostic. Nevertheless, it would 
be surprising if the rating scales from which they were 
derived could not be improved. Those familiar with the use 
of rating scales, particularly by personnel who are not 
trained in their use, will be well aware that the judges do 
not necessarily discriminate effectively on all of the char- 
acteristics (i.e., trainer features) described. 

Undoubtedly, many of the characteristics are correlated, 
i.e., they are not logically independent of one another. It 
is also possible, however, that the ratings of training de- 
vices could suffer from a sort of generali zed "halo" effect, 
a commonly observed phenomenon when supervisors rate their 
personnel on performance effectiveness. The "halo" effect 
occurs when one salient feature of a device (or individual) 
produces a generalized reaction, either negative or positive, 
such that the rater^s judgments of all other characteristics 
are biased in a similar direction by this feature. In the 
limiting case, the judgment of a single characteristic 
"colors" the judgment of all others, and only a single dimen- 
sion is required to extract all of the information obtainable 
from the rating scores. 

In the case of training devices it would be hoped that 
the users would judge each characteristic relatively inde- 
pendently of the others. This would be reflected by com- 
paratively low correlations among the ratings assigned to 
different features of the trainer. On the other hand, if 
there were a strong "halo" effect, the correlations would 
be very high. 

Throughout this report we have portrayed the results as 
if the simulation and training effectiveness factors were 
truly independent in the minds of the participants even 
though we knew that "halo" effects might have effectively 
reduced the number of dimensions on which the trainers were 
being judged. In the interest of improving the methodology, 
and the definition of the composite criterion of acceptance, 
a detailed statistical analysis was therefore performed in 
an attempt to better identify the nature and number of 
underlying dimensions on which the trainers were actually . 
judged. 
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Pearson product-moment correlations were computed for 
the scores of all trainers on the 22 simulation and training 
characteristics. The resulting correlation matrix was sub- 
jv'jcted to factor analysis, a technique for identifying the 
smallest number of common factors (dimensions) that account 
for all of the common variance in the matrix. Varimax rota- 
tion to oblique simple structure was performed in an effort 
to maximize the interpretabi lity of the results. 

The oblique primary factor loadings resulting from this 
procedure are shown in Table 21. For convenience, 10 factors 
were extracted which the computer program clearly indicated 
was more than enough to account for the common variance. 
Not all of the factors met conventional statistical criteria 
but it was evident that 7 of the 10 were clearly interpret- 
able on logical grounds. Each of these factors or under- 
lying dimensions is briefly described below. 

Factor 1. Recognized Value. The rating scale 
variables that loaded heavily on this factor 



were as f ol lows : 
I mportance of 

training opportunities provided .76 

Value of repeated use .69 

Level of training .68 
Use of trainer 

versus operational equipment .59 

Comprehensiveness oftraining .49 

Value of time in the trainer .38 



It seems evident that this factor is a rather direct 
reflection of the felt value of the training device in the 
overall scheme of training. It is a clear reflection of 
the fact that training devices are recognized as fulfilling 
a highly important need in addition to the opportunities for 
practice with operational equipment . This is undoubtedly a 
basic underlying consideration in the acceptance of any 
training device. 
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Factor 2, Tactical Realism. The features of a 
trainer that loaded most heavily on this factor 
had to do with simulated sensor performance, 
representation of targets, and weapon performance. 
All of the rating scale variables comprising this 
factor relate to the adequacy with which the tac- 



tical problem is presented. 

Simulation of senspr performance ,76 

Simulation of target evasion ,71 

Simulation of weapon firing ,58 

Simulation of displays ,48 



Factor 3. Provision for Evaluation. This factor 
was defined by only two rating scale character- 
istics : 

Immediacy of performance evaluation ,80 
Comp 1 et eness of 

performance evaluation • 77 

Although the definition of a "common" factor technically 
requires high loadings by more than two variables, these two 
characteristics clearly logically identify a basic dimension 
of trainers that is well defined in the minds of evaluators 
of training devices. No other characteristic of the trainer 
showed even a modest loading on this factor. 

Factor 4, Performance "Feel^n This factor was 
also a "doublet" with heavy loadings on two char- 
acteristics that are logically related in training 
devices. These were: 

Simulation of controls .75 

Simulation of performance ,67 

There were no other significant loadings on this factor. 
It is clearly a factor that is of maj or concern to us ers of 
operational flight trainers and other devices, such as sub- 
marine diving trainers, which are characterized by extensive 
feedback to the operator through the controls and apparent 
motions of the vehicle. 



6 
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Factor 5 • Operational Readiness . The trainer 
characteristics loading on this factor included 
both maintenance and operational considerations • 
Those with the heaviest loadings were as follows: 



Reliability and maintainability 



• 69 



Prob lem setup 



• 57 



Training software 



• 49 



This factor appears to reflect the "readiness" of the 
device to meet training needs from both a reliability and 
training convenience viewpoint. It undoubtedly reflects an 
important dimension in acceptance, both by students and by 
the personnel who must keep devices operational. 

Factor 6. Communication Provisions. The only 
rating scale variables loading heavily on this 
factor were the two items describing provisions 
for communications in the trainer. The load- 
ings were as follows: 



Factor 7 • Environmental Simulation. This is 
another example of a doub let that is logically 
well defined. In this case the only rating 
scale variables with heavy loadings had to do 
with environmental simulation • The loadings 
were as follows: 

Simulation of 

internal operating environment •SI 
Simulation of 

external operating environment ,43 

Factors 8, 9, and 10 showed appreciable loadings by only 
a single rated characteristic so no attempt at interpretation 
was made . It is interesting to note that all of the trainer 
characteristics included in the rating scale loaded heavily on 
at least one common factor with the exception of the last 
scale, "Comparison with Similar Trainers," The re as on s for 
this unexpected result are not clear. This particular scale 
was intended as a kind of summary evaluation after all of the 



Simulation of 
communication procedures 



.66 



Simulation of 
communication prob 1 ems 



• 58 
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other trainer characteristics had been considered. However, 
it loaded only ,22 on Factor 1 (Recognized Value) and .29 
on Factor 5 (Operational Readiness), Its loadings on other 
factors were negligible with the exception of Factor 8 (.33) 
which could not be defined. Possibly this overall comparison 
scale led to a complex set of considerations on the part of 
the raters which did not produce much reliable variance. Or, 
perhaps a major difficulty was that many of the respondents, 
being relatively new to the weapons system which the trainer 
represented, had insufficient past experience in comparable 
trainers. In either case, the value of this particular scale 
seems in doubt. 

In effect, the results of the factor analysis were to 
reduce the original 22 trainer characteristics included in 
the rating scale to just 7 basic dimensions: 



1 . 


Recognized Value 


2. 


Tactical Realism 


3. 


Provision for Evaluation 


4. 


Performance "Feel" 


5. 


Operational Readiness 


6. 


Communication Provisions 


7. 


Environmental Simulation 



If the rating scale were to be revised for future use, 
it could be argued that it need include only the 7 factors 
listed above since, generally speaking, those are all the 
dimensions that were discriminated by the raters. We would 
argue against this procedure except in the. case of Factor 1, 
Recognized Value, It would seem that some combining of the 
six variables loading on this factor could occur without 
serious loss of information. From the viewpoint of trainer 
acceptance, the three variables loading heaviest on this 
factor probably reflect the most important considerations. 

With respect to the other factors, however, it is doubt- 
ful whether combining could occur without serious loss of 
diagnostic information. For example, although four trainer 
characteristics loaded heavily on Factor 2, Tactical Realism, 
elimination or combining of any of those characteristics with 
any others might result in obscuring important deficiencies 
(or advantages) of a particular trainer. 
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SECTION VII 
SUMMARY AND CONCLUSIONS 

Factors influencing the level of acceptance of 16 complex 
training devices representing air, surface , and subsurface 
operations were studied by means of specially developed rating 
scales and questionnaires. A total of 326 Navy personnel, 
representing students, instructors , administrators , and main- 
tenance technicians, participated in the study. 

A composite criterion of trainer acceptance was developed 
that was comprised of ratings of trainer effectiveness, judged 
adequacy of simulation, volunteered statements concerning the 
merits and shortcomings of each trainer , percent utilization, 
and ratio cf hours-used to hours-requested. On the basis of 
this composite criterion, both highly accepted trainers and 
trainers with acceptance problems were identified. 

The value of training devices to the overall training 
process is widely recognized by Navy personnel. Training de- 
vices are particularly well accepted with respect to the 
opportunity they provide to supp lement operational training , 
especially the training they provide in certain procedures 
that are too costly or too hazardous to be practiced in the 
operational environment. They are appreciated for the effi- 
ciency they represent with respect to the time required to 
achieve skills initially, and for maintaining acquired skills 
through a program of repeated use. 

However, it is to be emphasized that no trainer is 
totally "accepted" or "unaccepted ." There is little doubt 
that the acceptance of training devices can be increased, 
with the consequence o f b oth improved training and greater 
cost effectiveness. The following general findings of the 
study are offered with these objectives in mind. 

SIMULATION FACTORS THAT INFLUENCE ACCEPTANCE 

1. Using the composite criterion of acceptance described 
earlier, highly accepted trainers were rated systematically 
higher than those with acceptance problems on 8 of the 10 
simulation characteristics studied. 

2. Highly accepted trainers were rated as particularly 
superior on simulation of vehicle performance, effects of 
controls, di splay of i nf ormati on , and sensor performance . 

3. The trainers as a group,, whether highly accepted or 
not, were rated below the considered level of impoi:tance on 
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three simulation characteristics: displayed information, 
sensor performance, and target reactions • 

4. All trainers were rated high on communication pro- 
cedures but almost all were rated low on communication 

p rob 1 ems . However , the s imulation of communication p rob 1 ems 
was not considered highly important by the participants. 
This may reflect an overly restrictive definition of com- 
munication problems in the rating scale used for the study. 

5. ^'Unaccepted" trainers were judged poorest in simu- 
lation of the external environment, vehicle or equipment 
performance , control effects, sensor performance, and weapon 
firing and control. 

6. Flight trainers appear to experience accept an ce 
difficulties, regardless of their other merits, if they are 
rated low on simulation of vehicle performance and/or control 
effects ("feel") . 

7. Weapons systems trainers appear to experience accep- 
tance difficulties, regardless of their other merits, if they 
are rated low on simulated information displays, sensor per- 
formance, or target reactions. 

8. Some acceptance problems appear to have developed 
because the simulated performance characteristics of the 
trainer have not kept pace with changes to the operational 
equipment. In some cases the trainer appears to have been 
pro grammed on the basis of "theoreti cal'' performance curves 
that are different from those achieved in the operating 
environment. 

9. Operational personnel tend to be very sensitive to 
the specific response characteristics of the vehicle that 
is simulated by the trainer. If the trainer does not react 
in a fashion very similar to the operational equipment it 
simulates, it is considered non-acceptable regardless of 
whether the trainer was intended to be a high fidelity simu- 
lator or not. Although it may be argued that important 
skills can nevertheless be acquired in such a trainer, the 
attitude of "response-specificity" is a major obstacle to 
acceptance . 

OTHER TRAINER FEATURES THAT INFLUENCE ACCEPTANCE 

10. The highly accepted trainers as a group were rated 
sys tematical ly h i gher than trainers with acceptance prob lems 
on all 12 training characteristics and situational factors 
studied. 
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11. Highly accepted trainers were rated superior par- 
ticularly with regard to problem setup, training software, 
and reliability. Trainers with acceptance problems were 
rated particularly low on training software, reliability, 
comprehensiveness of training, and the advantages of the 
trainer as opposed to the use of operational equipment for 
training. 

12. Considered as a group, the trainers in the study 
tended to be rated lowest in the areas of training software, 
completeness of performance evaluat i on , and immediacy of 
performance evaluat i on. 

13. The trainers as a group were rated highest with 
respect to the opportunity they provided to supplement 
operational training, value of time spent in the trainer, 
and value of repeated use. 

14. In several cases trainer acceptance appeared to 
have been greatly reduced by inadequate (or totally lacking) 
training software. In contrast, in a few cases, exceptionally 
good training software apparently had greatly enhanced trainer 
acceptance. 

15. Trainers with which serious maintenance prob 1 ems 
were associated tended to be unaccepted. There are four 
sources of maintenance problems frequently mentioned: 



a. Inadequate techni c a 1 documentation ; 

b. Long delays in obtaining spare parts; 

c. Difficulty in maintaining government- 
furnished operational components (GFE) ; 

d. Severe problems of access, particularly 
in tr ai ler-housed trainers. 



It is generally recognized that trainer maintenance 
personnel have done an exceptional job of coping with these 
prob lems • 

CHARACTERISTICS OF USERS THAT INFLUENCE ACCEPTANCE 

16. The level of user experience in the area of opera- 
tions represented by a trainer is a factor in the acceptance 
of some trainers. In general, if a trainer is seen as having 
simulation deficiencies, the more experienced personnel will 
be more critical than the less experienced personnel . In 
addition, some inexperienced personnel depreciate the value 
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of a trainer because, in their view, its use conflicts with 
the opportunity to do the "real" thing. 

17. Important capabilities of some trainers are not 
fully appreciated because of the assumption that "the fleet 
knows best" what its training needs are and how to use the 
trainer to meet them. A related problem is the lack of 
technical qualifications (and often inferior rank) of those 
who are called upon to critique performance in the trainer. 

18. The "not invented here" phenomenon, often seen as 
a symptom of resistance to hardware and procedural innova- 
tions , appears to have its coun terpart with respect to 
training devices in the form of a "not consulted here" re- 
action. Specific deficiencies, particularly in the area 

of simulation, are often considered the result of a failure 
to consult operational personnel. The magnitude of any felt 
deficiencies may be exaggerated by this reaction. 

A related problem is that NTDC and the various systems 
commands are frequently seen as unresponsive to perceived 
deficiencies, the need for updating or modernizing, the re- 
quirements for improved documentation, etc. 

CHARACTERISTICS OF INSTRUCTION THAT INFLUENCE ACCEPTANCE 

19. Quality of instruction (including evaluation of 
performance) is a major problem in trainer acceptance. In 
some cases this stems from the fact that the instructor's 
role is relegated to training devicemen who are not technical 
experts in the area of operation represented by the trainer; 
in others, it is due to the assignment' of the instructor's 
role to operational personnel who are not fully knowledgeable 
concerning the trainer's capabilities and/or the learning 
proces s . 

20. The absence of standard syllabi and graded series 
of problems is an obstacle to effective use of some trainers 
and therefore to acceptance. 

21. A problem in some trainers is that the instructor 
simply has too many stations to observe and cannot possibly 
evaluate the performance of all students adequately. This 
has a negative influence on acceptance. 

22. Many personnel responsible for trainer operation 
feel out-of-touch with what operational personnel feel to 
be their training needs. 

23. There appear to be no standard qualifications for 
instructor personnel for most trainers. In some cases. 
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personnel who are trained primarily in maintenance serve as 
instructors. There appears to be no well-defined program 
for training in device operation. 

24, Training devicemen have, in many instances, done 
a remarkable job of s e 1 f - instruct ion in learning how to 
operate the trainer. However, serious problems can occur 
when they are transferred and their responsibilities assumed 
by less experienced personnel. This can produce acceptance 
problems for appreciable periods of time, 

25, Unrealistic programming of problems (for example, 
illogically correlated malfunctions or the failure to pro- 
gram sequentially related ones) has an adverse effect on 
trainer acceptance , 

PATTERNS OF USE THAT AFFECT TRAINER ACCEPTANCE 

2G, Acceptance (or rejection) of a trainer is fre- 
quently in conflict with the original design objectives of 
the trainer. Most often, the trainer is used in a manner 
;chat is considerably less sophisticated than its designed 
capabilities. However, such trainers are often well 
"accepted," In other cases, particularly with flight train- 
ers, greater sophistication is expected by the users than 
was apparently intended by the designers. Such trainers 
experience considerable rejection by a significant proportion 
of users, 

27, When some characteristics of the problems presented 
in trainers (for example, tactical problems) are "too easy" 
as judged by experienced personnel, the trainer tends to 
regress towards the status of an element ary procedures 
trainer, regardless of the fact that it may have some very 
sophisticated capabi 1 i ties , 

28, The full capabilities of many trainers are not 
apprec,iated by a substantial proportion of users (or poten- 
tial users) because of the lack of knowledge among operating 
personnel of what the trainer can do. In a few cases, 

high ly motivated administrative personne 1 have effectively 
promoted the trainer's use. In others, no one has assumed 
this responsibility. In many cases, the two-way communica- 
tion link that should relate fleet training needs to trainer 
capabilities is in need of considerable strengthening, 

RECOMMENDATIONS 

In the interest of improving the acceptance and utiliza- 
tion of training devices, the following actions are recommended. 
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1. The acceptance prof i ling technique developed for 
this study should be applied periodically to all complex 
trainers. Data on simulation, training characteristics, 
and situational factors influencing acceptance should be 
accumulated so that a body of knowledge is developed that 
will be useful for design decisions. Longitudinal studies 
should be conducted to determine how sensitive the technique 
is to changes in the level of acceptance. The reactions of 
Navy personnel to this survey indicates that they would 
readily cooperate in such a program. 

2. Effort should be made to increase acceptance and 
methods of utilizing complex trainers by increased invest- 
ments in training software. Detailed instructional guides, 
prepared by personnel who are both operationally knowledge- 
able and sophisticated in training methodology, should be 

a standard item for delivery with all complex trainers. 

3. Corresponding efforts shou Id be made to ameliorate 
the acceptance problems generated by inadequate technical 
documentation, spare parts, and troublesome GFE items that 
adversely influence trainer maintenance. 

4. Research should be conducted into the general prob- 
lem of trainer acceptance as a function of the "feel" of the 
simulated vehicle. The fundamental issue, of course, is the 
transferability of responses from the trainer to the opera- 
tional equipment. Rightly or wrongly. Navy personnel feel 
the need for a high level of specific response realism in 
operational trainers (i.e., they feel the trainer should 
react like a specific model of an aircraft, submarine, etc.). 
Their reactions clearly reflect a concern for the possibility 
of negative transfer. Probably this concern is justified 

for some types of trainers and not others. Research should 
be conducted to define design guidelines in this area and 
to better relate them to the phenomena of transfer. 

5. Similarly, research should be conducted into the 
necessary fidelity of simulation for tactics-related 
training such as sensor capabilities, information displays, 
target behavior, and weapon reactions. The problem is to 
decide what levels of simulation are required for effective 
training of different membeva of the tactical team for such 
tasks as target detection, classification, tracking, local- 
ization and attack. 

6. The problem of developing uniform and high standards 
of qualification for instructors needs to be solved, along 
with effective ways to train these personnel. The Navy should 
seriously reconsider its traditional policy of assuming that 
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operational personnel can identify their own training short 
comings and act as their own instructors in solving them, 

?• For every training device representing a major in- 
vestment, consideration should be given to the development 
of a ^'trainer advocate'* who, by virtue of his related opera 
tional experience, knowledge of the trainer's capab i 1 i t ies , 
limitations, and design objectives, and understanding of 
good training practice, can effectively close the communi- 
cation gaps between trainer users, designers, operational 
commands, and systems commands. 
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NAME OF RATER: 



FACTORS IN TRAINING DEVICE ACCEPTABILITY 



PART I: RATING SCALES 



INTRODUCTION 



Human Factors Research, Inc., is under contract to the 
Navy to determine why some training devices are highly- 
accepted by those who use them, and others develop an un- 
favorable reputation. This can be a very serious problem, 
since a trainer typically represents a large investment of 
money, and since the effectiveness of the trainer might be 
Influenced by its acceptability. 

One approach we are using to study this problem is to 
get information directly from the users of several training 
devices. For purposes of comparison, some of these trainers 
we are studying are highly accepted, while others may not be. 

The data we wish to obtain from you will require two 
stages. First, this set of rating scales will allow you to 
give evaluations of several important characteristics of 
the trainer in a quick and easy form. Second, there is a 
questionnaire for you to record some detailed reactions to 
the training device, its effectiveness, and its acceptance. 
In addition, a member of the research staff may contact you 
for further detail or clarification. 

In order to match up the rating scales, questionnaires, 
and interview notes, you are asked to put your name above, 
on the line vhich says ''NAME OF RATER." Please feel free 
to answer tho questions l^onostly and frankly. Our report 
to the Kavy will be in terms of trends and averages; only 
our company rcsoarch staff will see the actual rating scales 
and interview notes. 
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(Sample page from rating scale illustrating format) 
SIMULATION OF SENSOR PERFORMANCE 



How realistic is the simulation of sensor equipment in the trainer? Consider such 
factors as the physical appearance of objects in the environment and the environ- 
ment itself as displayed to the eyes or ears; the nuraber and variety of objects 
simulated; how detectability is affected by the surrounding environment. If applica- 
ble, consider how well different classes of targets are simulated. 



15 T 
14 
13 

12 -- 
11 -- 
10 
9 -- 
8 -- 
7 

6 .- 
5 ■- 
4 .. 
3 .- 
2 .- 
1 .- 
0 ■- 



The simulated presentations cannot be distinguished from the way 
objects or targets appear in the operating environment. 



The simulated presentations arc somewhat different from operational 
equipment, but training is not seriously limited. 



The simulated presentations are sufficiently different from operational 
equipment that some functions cannot be trained effectively, or train- 
ing Is generally impaired. 



The simulated presentations arc so different from operational equip- 
ment that what is learned in the trainer could have a negative effect 
on operational performance. 



[ ] NOT APPLICABLE [ ] INSUFFICIENT KNOWLEDGE OR EXPERIENCE 
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(NOTE: The questions on the following pages were asked of 
all qualified respondents. In the interest of saving space, 
the format of the questionnaire has not been reproduced.) 



FACTORS IN TRAINING DEVICE ACCEPTABILITY 
PART II: QUESTIONNAIRE (TRAINEES) 



Please fill out PART I: RATING SCALES, if you have 

not already done so, before going on to this question- 
naire. 

The form of this questionnaire is intended to 

encourage somev;hat detailed responses to the questions, 

without taking a great deal of time. Try to give enough 

detail so that the research staff can clearly understand 
your ansvvers. 

If you wish to make comments about the trainer or 
about your reactions to it, but these comments do not fit 
any of the questions specifically asked, please write 
them on the last page of this questionnaire. We welcome 
such comments, since they can point to unexpected factors 
in trainer acceptance and effectiveness. 

Please notice that" there are questions on both sides 
of each page. Experience indicates that the questionnaire 
can be completed in about an hour. 
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INTERVIEW SCHEDULE 
(ADMINISTRATIVE AND INSTRUCTOR PERSONNEL) 

Name 

Rank/Rate^ Length of Service: Yrs. Mos . 

Present Billet 

How long? Yrs. Mos. 

Educational level--hi ghest grade in school: 

7 8 9 10 11 College degree? [ ] Yes [ ] No 
12 13 14 15 16 Graduate study? (Years) 

Navy schools related to present billet: 



Area of specialization in the Navy: 



Relation to this training device; 

[ ] Administration [ ] Instructor [ ] Trainee 
[ ] Maintenance [ ] Other 

Details: 

How long have you been involved with this device? 

Yrs. . Mos. 
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Y !• Are the personnel scheduled into this trainer all essen- 

tially the same in background, experience, previous 
training, and so forth, or are there important differences 
among the trai nees? 

2, What are the differences? 

3, In what ways is the trainer used differently for different 
types of trai nees ? 

4, Is student use of this trainer voluntary or is it scheduled 
by authority? 

5, Do you think the amount of time scheduled for individual 
students or teams is too much, too little, or about right? 

6, Why do you feel this way? 

7, How often do students use it or try to use it? 

8, Is this often enough to satisfy student training needs? 

9a, Why don't students use the trainer enough to satisfy their 
training needs? 

/' 9b, What particular training needs do you feel this device 

^ sati sf ies? 

10* In general, when students come into this trainer, are they 
adequately prepared to get maximum benefit from using the 
trai ner? 

11. For personnel who would benefit by using this trainer, 
are the administrative procedures for gaining access to 
it easy? 

12* What are the administrative obstacles? 

13. Do you know of any administrative obstacles at all? 

14, Is the number of trainers or their physical location a 
factor in utilization? 

15. Do problems in getting a team together influence its use 
or its effectiveness? 

16, Approximately what is the utilization rate for this trainer? 

17a« What would you say is the biggest hindrance to maximum 
uti 1 i zation? 

17b* What other factors Interfere with maximum utilization? 
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18. 

19. 
20. 

21 . 
22. 

23. 

24. 
25. 

26. 
27. 

27a. 

27b. 

28. 
29. 

30. 
31 . 



In general are the instructors who operate the trainer 
well qualified to operate it? 

Why do you say that? 

Are they well qualified in the ^JkiUs which the ^.rainer 
trai ns? 

Why do you say that? 

Do you think it's important that they be skilled in these 
skills? 

In general do you think the instructors feel that the 
trainer does an adequate job of training? 

Do you know why they feel that way? 

Are the instructors well qualified to evaluate student 
performance in the trainer? 

Why do you say that? 

On what kind of data is the evaluation based; for instance, 
does it depend on the instructors' memory, informal notes, 
a checklist, a computer read-out, or what? 

What kind of information does the trainee get, and at what 
points during his training is he evaluated? Do the evalua- 
tions pertain to team performance or individual performance? 

Is the evaluation of performance in the trainer reported to 
anyone other than the students being trained? If so, to 
whom is the report made, and what is done with the in- 
formation? 

How could the evaluation be Improved to be more useful to 
the students? 

We're trying to find out what people are likely to hear 
about this trainer before they use it. What favorabl e or 
unfavorable comments did you hear before you had direct 
experience with it? 

From whom, and where? 

Based on your experience with the trainer, what would you 
be likely to say to someone else about it, both pro and 
con? 
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32. As far as you know, does the trainer have any kind of a 
reputation with regard to maintenance or reliability? 
For instance, is it regarded as a generally reliable or 
generally unreliable machine? I'd like to know if the 
trainer has a reputation in this respect, even though it 
might be different from your own opinion. 

33. What reputation does it have with regard to maintenance 
or rel iabi 1 i ty ? 

34. With whom does it have this reputation? 

35. To your knowledge, has it ever been necessary to cancel 
or delay scheduled training due to equipment casualties? 

36. How many times has this happened to you ? 

37. How long were these delays? 

38. Are there any particular maintenance problems that you're 
aware of from your own personal knowledge? 

39. Are there any particular maintenance problems that you've 
become aware of through scuttlebutt? 

40. Do you know of any person or group of people at the present 
time who are actively promoting the use of this trainer? 

41 . Who are they? 

42. What are they doing to promote the trainer? 

43. What positive or negative effects would you say these 
efforts are having on the use of the trainer or on peoples' 
attitudes or knowledge about the trainer? 

44. To your knowledge, are there any people now, or have there 
been any in the past, with an unfavorable opinion of the 
trainer? 

45. Who are they? 

46. Why do they feel this way? 

47. To what extent and in what ways has this negative opinion 
spread to other people? 

48. During the development of this trainer, did the designers 
obtain or try to obtain information from the users of 
similar devices (either instructors, trainees, or other 
Navy personnel) to assist them in making design decisions? 
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49. Considering all the characteristics of this trainer, the 
various features that were designed into it, the way it 
operates, etc., would you guess that users of similar 
devices were consulted to assist in making design decisions? 

50a. With regard to the devel opment of this trainer, do you 
know where and how the requirement for it originated? 

50b. What are the training purposes for which this trainer was 
originally designed? I'd like to know what the designers' 
original purposes were, even if the trainer is being used 
differently now. 

50c. Considering the training needs of the trainees, what 

limitations are there in what this device was designed 



51. Would you say that the present uses of the trainer are 
exactly the same as what you've said the designers in- 
tended, somewhat different, or very different? 

52. Are there some skills that cannot be trained in relation 
to the training requirement it was supposed to fulfill? 

53. Is the trainer being used to train skills other than those 
originally intended by the designers? 

54a. Coul d the trainer be used effectively to train skills other 
tFan those for which it's presently being used? 

54b. Shoul d it be used in these other ways, either i nstead of 
the way it's being presently used, or ^n addition to its 
present uses? 

55. Consider the present uses of the trainer. Could the same 
training be accomplished with a less complex device? 

56. To your knowledge, have any hardware modifications been 
made, either official or unofficial, since the installa- 
tion of this trainer? 

57. What modifications have been made? 

58. By what agency or group were they initiated? 

59. Have these modifications affected the trai ni ng capabilities 
of the trainer? 

60. Have these modifications allowed more functions to be 
trained, restricted the original range of functions, or 
had some other effect? 



to do? 
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61. In what specific ways have the training functions of the 
device been changed by the modifications? 

62. Would you judge the overall effect of the modifications 
to be good or bad, as far as training effectiveness is 
concerned? 

63. What other effects have these modifications had, for 
instance, on reliability and maintenance, problem set-up, 
and operation? 

64. Are there any channels of communication that you can use 
to make suggestions for improving this trainer? 

65. What are they? 

66. Does anything effective happen if these channels of com- 
munication are used? In other words, do you feel anything 
is done about legitimate complaints and suggestions? 

67. Have you ever registered any such complaints or suggestions? 

68. Do you ever feel frustrated about the way suggestions or com- 
plaints are handled? 

69. What would you like to see in the way of new lines of 
communication or improvements in existing lines of com- 
muni cation? 

70. Have you had experience with other trainers in this area 
of operations? Which ones? 

71. On the whole, how worthwhile do you think trainers of this 
type are? Why? 

72. Overall, in terms of actually accomplishing effective 
training, what would you say are the most serious defi- 
ciencies of trainers of this type? Please start with 
the deficiency you think is most serious, and give me 
some indication of why you feel that way. 

73. If you were in a position to suggest design changes for a 
new model of this trainer, what design changes would you 
suggest to improve training effectiveness? 

74. What design changes would you suggest to make the trainer 
more cost-effective? 

75. What design changes would you suggest to make it easier to 
provide students with more complete or more meaningful 
evaluations? 
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76. In order to provide better evaluations to the students, 
what kind of information would you like the trainer to 
provide you with? 

77, This is the end of the interview, but before you go Pd 
like to give you a chance to make any comments you'd 
like to about the trainer. Perhaps we didn't cover some- 
thing that you consider important, or perhaps something 
we talked about needs further explanation. Do you have 
any comments you want to make? 
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listing, the source of which is updated on a weekly 
basis. It is not practical to prepare distribution 
lists each time labels are prepared. Therefore, such 
lists are prepared semiannually, and a slight discrep- 
ancy may exist between the addressees on this list and 
those appearing on the labels used to distribute this 
publication. 



USAF HUMAN RES LAB 
AFHRL-DO OPERATION OFC 
VROO^S AFB TX 78235 

USAF HUMAN RES LAB 
AFHRL-TT 
TECH TRNG DIV 
LWRY AFB CO 80230 

USAF HUMAN RES LAB 
PERSCftlNEL RSCH DIV 
UCKLAND AFB TX 78236 

USAF HUMAN RES LAB 

AFHRL-TR 

TRNG RES DIV 

WRIGHT -PATTERSON AFB 

OH 45433 

CHIEF NAVAL RESEARCH 
PSYCHOLOGICAL SCIENCES 
CODE 450 NAVY DEPT 
ARLINGTON VA 22217 

DR RALPH R CANTER 
DIR MIL MANFWR RSCH 
OASD M-RA MR-U 
PENTAGON RM 3*960 
WASHINGTON DC 20301 

NATL SCIENCE FDN 
ATTN DR HENRY S ODBERT 
1800 G ST NW 
WASHINGTON DC 20550 

CHIEF NAVAL OPERATKMS 
ATTN DR J J COLLINS 
0P.-07T16 NAVY DEPT 
WASHINGTON DC 20350 



I DR JOHN MEYER 

HDQTRS AIR TRNG CMD XPT 
RANDOLPH AFB TX 78148 

I USAF HUMAN RES LAB 

AFHRL-FT 
FLYING TRNG DIV 
WILLIAMS AFB AZ 85224 

1 CHIEF NAVAL MATERIAL 

MAT 03 IM 

WASHINGTON DC 20360 

I JOSEPH J COWAN 

CH PERS RSCH BRANCH 
USOG HQ PO-l STA 3-12 
400 SEVENTH ST S W 
WASHINGTON DC 20226 

I BUREAU NAVAL PERSONNEL 

ATTN PERS A3 
ARLINGTON ANNEX 
WASHINGTCm DC 20370 

I DEPT OF PSYCHOLOGY 

UNIV OF IL AT 
URBANA-CHAMPAIGN 
PROF PSYCH J A ADAMS 
CHAMPAI^ IL 61820 

I CHIEF NAVAL OPERATIONS 

ATTN CDR H J CONNERY 
MSG OP-701E2 NAVY DEM 
WASHINGTON DC 2035 ) 

I INSTITUTE OF AVIATION 

UNIV OF IL 
URBANA-CHAMPAIGN 
SAVOY IL 61874 
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CHIEF NAV OPERATICWS 
ATTN M K MALEHORM 
0P-14C NAVY DEPT 
WASHINGTON DC 20350 



COMV'^NDER 
NAVdnxPSSYSCMD 
CODE 03 

WASHINGTON DC 20360 



COMMANDER 
NAVELECTSYSCMD 
CODE 03 

WASHINGTON DC 20360 

NAV AEROSPACE MED INST 
NAV AEROSPACE MED CTR 
ATTN CH AVIA PSYCH DIV 
PENSACOLA FL 32512 



CDR NAVSHIPS SYS CMD 
NAVSHIPS SYS CMD HQS 
SHIPS 03H GEO N GRAINS 
WASHINGTON DC 20360 

COMMANDER 
NAVAIRSYSCMD 
CODE 03 

WASHINGTON DC 20360 



COMMANDER 
NAVSUPSYSCMD 
CODE 03 
WASHINGTON DC 



20360 



CO NAVMED RSCH INST 
NATL NAVMED CTR 
ATTN TECH REF LIB 
BETHESDA MD 20014 



COMMANDER 
NAVORDSYSCMD 
CODE 03 
WASHINGT(»9 DC 



20360 



HUMAN RES RSCH ORG 
DIV NO I SYS OPER 
300 N WASHINGTON ST 
ALEXANDRIA VA 22314 



SCIENTIFIC TECHNICAL 
INFORMATION OFFICE 
NASA 

WASHINGTON DC :;0546 

BEHAVIORAL SCI DIV 
QFC CHIEF RSCH - DEVEL 
DEPT OF ARMY 
WASHINGTON DC 20310 



CO NAV MISSILE CTR 
ATTN HD HUMAN FACTORS 
SNCRG BR 

POINT MUGU CA 93042 



HUMAN RESOURCES RESCH 
ORGZTN DIV 6 AVIATICW 
P 0 BOX 428 
FORT RUCKER AL 36360 



CH RSCH AND DEVLPMT 
CH BEHAVIORAL SCI DIV 
DEPT OF ARMY 
WASHINGTON DC 20310 



HQS AF SYS CMD DLSL 
OFC SCIENTIFIC RSCH 
ANDREWS AFB 
WASHINGTOi DC 20331 



COMMANDING OFFICER 
NAVY MEDICAL 
NEUROPSYCHIATRIC RESCH 
UNIT 

SAN DIEGO CA 92152 



COMMANDER 

NAVFAC ENGR CMD 

NAVFAC ENGR CMD HDQ 

ASST DIR R6tD DIV(CODE 03) 

WASHINGTON DC 20390 



DIR DEF RSCH - ENGRG 
ARPA 

BEHAVIORAL SCI DIV 
ATTN LCOL A W KIBLER 
WASHINGTON DC 20301 



DIRECTOR 

US ARMY MOTIVATION & TNG LAB 
COMMONWEALTH BLDG 
1300 WILSON BLVD 
ARLINGTW VA 22209 



ERLC 
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COMDT OF MARINE CORPS I 
CODE AC3C 

WASIUNGTCm DC 20380 

CHIEF NAVAL RESEARCH I 
ATTN CODE 458 DEFT NAV 
ARLINGTON VA 22217 

COMMANDING OFFICER I 
PBRS-TRNG RSCH DEV lAB 
SAN DIEGO CA 92152 

US ARMY BEHAVIORAL SCI I 
RESEARCH LAB 
COMMONWEALTH BLDG 
RM 239 

1320 WILSON BLVD 
ARLINGTON VA 22209 

EXECUTIVE EDITOR I 
PSYCHO ABSTRACTS 
AMERICAN PSYCH ASSOC 
1200 ITTH ST NN 
WASHINGTON DC 20036 

NAV PERSONNEL RSCH & I 

DEVBLCVMENT LAB 

ATTN LIBRARY 

BLDG 200 RM 3307 WNY 

WASHINGTON DC 20390 

CHIEF OF NAVAL TECH TRNG I 
NAVAL AIR STATI(X«, MEMPHIS (75) 
MILLINGTON, TN 38054 
(ATTN: DR. LARRY HARDING, 
CODE 34) 

DEFENSE DOCUMENTATION CENTER 12 
Cameron Station 
Alexandria, VA 22314 

Naval Training Equipment Center 75 
Orlando, Florida 32813 



DIR HUMAN RES RSCH ORG 
300 N WASHINGTON ST 
ALEXANDRIA VA 22314 

CHIEF OF NAVAL TRNG 
CODE 017 NAS 
PENSACOLA FL 32508 

CDR NAVAIR DEVEL CTR 
ATTN HUMAN ENGRG BR 
WARMINISTER PA 18974 

DR JOHN W WEISZ 
DIR HUMAN ENGRG LAB 
USA ABERDEEN RSCH 
DEVEL CTR 

ABERDEEN PROV GROUNDS 
MD 21005 

CO NAVAIR TECH TRNG 
NAS MEMPHIS 
ATTN DR G D MAYO 
HD RESEARCH BR 
MILLINGTON TN 38054 

ERIC CLEARINGHOUSE ON 
EDUCATION MEDIA— TECH 
STANFORD UNIV 
STANFOU) CA 94305 
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